What is research? Why do we conduct it?

by

An important question about discovering some new knowledge is What is Research? Why do we do Research? The answer of What is research and how it is conducted is explained below.

What is Research

Research is an inquiry. It is a process of discovering some new knowledge, that involves multiple elements such as theory development and testing, empirical inquiry, and sharing the generated knowledge with others such as experts and colleagues. A short description of the elements of theory is:

The theory is a set of ideas and perceptions that helps people to understand complex concepts and the relationships among these concepts. To develop and/or test a theory, researchers conduct empirical inquiries, collect and analyze relevant data, and discuss the findings from empirical results. Once theories have been through the research process, it is necessary to share the results of the studies with others such as researchers (related to the study) present papers at conferences, and publish reports in journals and other publications.

There are two ways to use the results of a study:

  1. The results may contribute to researchers’ general understanding of the topic they have researched i.e. studied and may contribute to, understanding how the economy works, why price inflation happens, which factors increase a candidate’s chances of winning an election, etc. The generalizations of results that researchers draw from their studies on these issues can be shared with other researchers and the general public to advance society for the understanding of the topic.
  2. The results of a study may contribute to solving particular problems in a nation, state, or community. For example, a study on the healthcare needs of the elderly in a community may discover that their primary need is finding vehicles for transportation when they want to visit their doctors. The leaders of the community (such as the mayor, and city council) may use this information from the healthcare study, to allocate some money for the transportation needs of the elderly in the next year’s budget.

Therefore, research is a tool that builds blocks of knowledge that in turn contribute to the development of science.

What is Research, Why we Conduct a Research?

Why conduct research?

  • To understand a phenomenon, situation, or behavior under study.
  • To test existing theories and to develop new theories based on existing ones.
  • To answer different questions of “how”, “what”, “which”, “when” and “why” about a phenomenon, behavior, or situation.
  • Research-related activities contribute to forming (making) new knowledge and expanding the existing knowledge base.

High-Quality Research

Nowadays one can collect/ gather information about almost anything from the Internet Just do a Google search. But a question is, does every Google search good research? Not quite! Do remember, though you will find some of the information, it may or may not be valid or high-quality information. A lot of the information available on the Internet is good and useful, but some are not. There may be misinformation too on the Internet. The information you find on the internet may be someone’s pure opinion, have some fabrication in it, or be based on some unsystematic research or unauthentic information. In short, the information may be valid (objective, true).

Therefore, a high-quality research project:

  • is based on the scholarly work that has been already done by others in the field,
  • can be replicated/ reproduced,
  • is a generalization to other settings,
  • is based on some logical rationale and tied to other existing theory;
  • is doable and can be done practically, i.e. when deciding the scope of research. A researcher should consider the availability of time and resources,
  • generates some new questions,
  • is incremental,
  • is an apolitical (politically neutral) activity that should be undertaken for the betterment of society.

Two Types/ Purposes

Typically, there are two types/purposes: Basic Research and Applied Research

  1. To find out about truths regarding human behaviors, societies, economy, etc., or to understand them better. This type is called basic research.
  2. To answer practical questions and support making informed decisions. This type is called applied research.

Note that, most of the public administration and public policy research projects are of the second kind.

Learn about Qualitative vs Quantitative Research

Learn R Programming Language

Model Selection Criteria (2019)

by

All models are wrong, but some are useful. Model selection criteria are rules used to select a (statistical) model among competing models, based on given data.

Several model selection criteria are used to choose among a set of candidate models, and/ or compare models for forecasting purposes.

All model selection criteria aim at minimizing the residual sum of squares (or increasing the coefficient of determination value). The criterion Adj-$R^2$, Akaike Information, Bayesian Information Criterion, Schwarz Information Criterion, and Mallow’s $C_p$ impose a penalty for including an increasingly large number of regressors. Therefore, there is a trade-off between the goodness of fit of the model and its complexity. The complexity refers to the number of parameters in the model.

Model Selection Criteria

Model Selection Criteria: Coefficient of Determination ($R^2$)

$$R^2=\frac{\text{Explained Sum of Square}}{\text{Total Sum of Squares}}=1-\frac{\text{Residuals Sum of Squares}}{\text{Total Sum of Squares}}$$

Adding more variables to the model may increase $R^2$ but it may also increase the variance of forecast error.
There are some problems with $R^2$

  • It measures in-sample goodness of fit (how close an estimated $Y$ value is to its actual values) in the given sample. There is no guarantee that $R^2$ will forecast well out-of-sample observations.
  • In comparing two or more $R^2$’s, the dependent variable must be the same.
  • $R^2$ cannot fall when more variables are added to the model.

Model Selection Criteria: Adjusted Coefficient of Determination ($R^2$)

$$\overline{R}^2=1-\frac{RSS/(n-k}{TSS(n-1)}$$

$\overline{R}^2 \ge R^2$ shows that the adjusted $R^2$ penalizes for adding more regressors (explanatory variables). Unlike $R^2$, the adjusted $R^2$ will increase only if the absolute $t$-value of the added variable is greater than 1. For comparative purposes, $\overline{R}^2$ is a better measure than $R^2$. The regressand (dependent variable) must be the same for the comparison of models to be valid.

Model Selection Criteria: Akaike’s Information Criterion (AIC)

$$AIC=e^{\frac{2K}{n}}\frac{\sum \hat{u}^2_i}{n}=e^{\frac{2k}{n}}\frac{RSS}{n}$$
where $k$ is the number of regressors including the intercept. The formula of AIC is

$$\ln AIC = \left(\frac{2k}{n}\right) + \ln \left(\frac{RSS}{n}\right)$$
where $\ln AIC$ is natural log of AIC and $\frac{2k}{n}$ is penalty factor.

AIC imposes a harsher penalty than the adjusted coefficient of determination for adding more regressors. In comparing two or more models, the model with the lowest value of AIC is preferred. AIC is useful for both in-sample and out-of-sample forecasting performance of a regression model. AIC is used to determine the lag length in an AR(p) model also.

Model Selection Criteria: Schwarz’s Information Criterion (SIC)

\begin{align*}
SIC &=n^{\frac{k}{n}}\frac{\sum \hat{u}_i^2}{n}=n^{\frac{k}{n}}\frac{RSS}{n}\\
\ln SIC &= \frac{k}{n} \ln n + \ln \left(\frac{RSS}{n}\right)
\end{align*}
where $\frac{k}{n}\ln\,n$ is the penalty factor. SIC imposes a harsher penalty than AIC.

Like AIC, SIC is used to compare the in-sample or out-of-sample forecasting performance of a model. The lower the values of SIC, the better the model.

Model Selection Criteria: Mallow’s $C_p$ Criterion

For Model selection the Mallow criteria is
$$C_p=\frac{RSS_p}{\hat{\sigma}^2}-(n-2p)$$
where $RSS_p$ is the residual sum of the square using the $p$ regression in the model.
\begin{align*}
E(RSS_p)&=(n-p)\sigma^2\\
E(C_p)&\approx \frac{(n-p)\sigma^2}{\sigma^2}-(n-2p)\approx p
\end{align*}
A model that has a low $C_p$ value, about equal to $p$ is preferable.

Model Selection Criteria: Bayesian Information Criteria (BIC)

The Bayesian information Criteria is based on the likelihood function and it is closely related to the AIC. The penalty term in BIC is larger than in AIC.
$$BIC=\ln(n)k-2\ln(\hat{L})$$
where $\hat{L}$ is the maximized value of the likelihood function of the regression model.

Cross-Validation

Cross-validation is a technique where the data is split into training and testing sets. The model is trained on the training data and then evaluated on the unseen testing data. This helps assess how well the model generalizes to unseen data and avoids overfitting.

Note that no one of these criteria is necessarily superior to the others.

Read more about Correlation and Regression Analysis

Learning R Language Programming

Google Search Tricks and Tips

by

Here are some of the most useful Google Search Tricks and Tips that can be used in Google from basic tips to new features. Let us start with helpful Google Search Tricks and Tips that can be useful for search queries by mathematicians and statisticians.

Google Search Tricks and Tips

Google Search Tricks and Tips 1: Double Quotes (for Exact Search)

The use of double quotes yields only the pages with the same words in the same order (containing a specific phrase) as what’s in the quotes.

Google Search Trick 2: Asterisk within Quotes (to Specify Unknown Words)

Searching a phrase in double quotes with an asterisk will search all variations of that phrase. For example “* matrix in regression analysis” will yield pages that have different words before and after ‘matrix in regression analysis’ such as “hat matrix in regression analysis”, “the inverse-partitioned-matrix method in linear regression analysis”, “a matrix form, in regression analysis”, and “second important matrix in regression analysis” etc.

Google Search Trick 3: Minus Sign to Exclude Words from Search

If you want to exclude (eliminate) certain words from your search, you can use the minus sign. For example, the “hat matrix in regression -outlier” will result in all the pages related to the hat matrix in regression but will not contain outlier words in the searches.

Google Search Trick 4: Tilde symbol (~) to Search for Similar Words

The tilde symbol used in the phrase will search for a word and all its synonyms. For example, ~Cross Table will result in crosstable, crosstabulation, cross-table, and cross-table query, etc.

Google Search Tricks and Tips 5: OR Operator for Multiple Words Searching

The “OR operator searches the pages that include either word before and after the OR operator. For example, residuals or error will result in the words “residuals” and either “error”.

Google Search Trick 6: Numerical Range

The use of a numerical range of numbers results in pages that match these numbers. For example, “History of Statistics 2000…2019”.

Google Search Tricks and Tips 7: Finding the Meanings of Word or Phrase

The define keyword is used to define a word or phrase. For example, define statistics, define: define: residuals or error, and define: goodness of fit test

Google Search Trick 8: Search a Particular Website

The site: function searches a particular website. For example, Learning statistics site:edu will result in pages found on .edu websites. The other examples can be “Learning statistics site: com”, and Learning statistics site:itfeature.com, etc.

Google Search Trick: Search a particular website

Google Search Trick 9: Search Webpages Linked to a Particular Website

The link: function searches for web pages that link to a particular website. For example, link: itfeature.com, link: stat.bzu.edu.pk

Google Search Trick 10: Math Answers

Google performs basic math functions for example, 4.7, 30% of 55, 20^2, sqrt(4), exp(4), log(10), cos(90), etc.

Google Search Trick 11: Unit Conversion

Converts the units of a measure. For example, 5cm in the foot, 100$ in PKR, 42 days in a fortnight, 10 mph in the speed of light, 100 miles in leagues, and 100 Km in miles, etc.

Google Search Trick 12: Compare using “vs”

A one-by-one comparison can be searched using the “vs” keyword. For example, statistics vs parameters, descriptive vs inferential statistics, AIC vs BIC, and statistics vs mathematics.

https://itfeature.com statistics help

R Faqs: Frequently Asked Questions

Online MCQs Website with Answers

Job Interview Recently Asked Questions