{"id":109036,"date":"2021-02-17T17:20:55","date_gmt":"2021-02-17T11:50:55","guid":{"rendered":"https:\/\/www.vskills.in\/certification\/tutorial\/?page_id=109036"},"modified":"2024-04-12T14:30:23","modified_gmt":"2024-04-12T09:00:23","slug":"factor-analysis","status":"publish","type":"page","link":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/","title":{"rendered":"Factor Analysis"},"content":{"rendered":"\n

Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. For example, it is possible that variations in four observed variables mainly reflect the variations in two unobserved variables. Factor analysis searches for such joint variations in response to unobserved latent variables. The observed variables are modeled as linear combinations of the potential factors, plus “error” terms. The information gained about the interdependencies between observed variables can be used later to reduce the set of variables in a dataset. Computationally this technique is equivalent to low-rank approximation of the matrix of observed variables. Factor analysis originated in psychometrics and is used in behavioral sciences, social sciences, marketing, product management, operations research, and other applied sciences that deal with large quantities of data.<\/p>\n\n\n\n

Factor analysis is related to principal component analysis (PCA), but the two are not identical. Latent variable models, including factor analysis, use regression modeling techniques to test hypotheses producing error terms, while PCA is a descriptive statistical technique. There has been significant controversy in the field over the equivalence or otherwise of the two techniques.<\/p>\n\n\n\n

The key concept of factor analysis is that multiple observed variables have similar patterns of responses because they are all associated with a latent (i.e. not directly measured) variable. For example, people may respond similarly to questions about income, education, and occupation, which are all associated with the latent variable socioeconomic status.<\/p>\n\n\n\n

In every factor analysis, there are the same number of factors as there are variables.  Each factor captures a certain amount of the overall variance in the observed variables, and the factors are always listed in order of how much variation they explain.<\/p>\n\n\n\n

The eigenvalue is a measure of how much of the variance of the observed variables a factor explains.  Any factor with an eigenvalue \u22651 explains more variance than a single observed variable.<\/p>\n\n\n\n

So if the factor for socioeconomic status had an eigenvalue of 2.3 it would explain as much variance as 2.3 of the three variables.  This factor, which captures most of the variance in those three variables, could then be used in other analyses.<\/p>\n\n\n\n

The factors that explain the least amount of variance are generally discarded.  Deciding how many factors are useful to retain will be the subject of another post.<\/p>\n\n\n\n

Factor Loadings<\/strong><\/p>\n\n\n\n

The relationship of each variable to the underlying factor is expressed by the so-called factor loading. Here is an example of the output of a simple factor analysis looking at indicators of wealth, with just six variables and two resulting factors.<\/p>\n\n\n\n

Variables<\/strong><\/td>Factor 1<\/strong><\/td>Factor 2<\/strong><\/td><\/tr>
Income<\/td>0.65<\/td>0.11<\/td><\/tr>
Education<\/td>0.59<\/td>0.25<\/td><\/tr>
Occupation<\/td>0.48<\/td>0.19<\/td><\/tr>
House value<\/td>0.38<\/td>0.60<\/td><\/tr>
Number of public parks in neighborhood<\/td>0.13<\/td>0.57<\/td><\/tr>
Number of violent crimes per year in neighborhood<\/td>0.23<\/td>0.55<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

The variable with the strongest association to the underlying latent variable. Factor 1, is income, with a factor loading of 0.65. Since factor loadings can be interpreted like standardized regression coefficients, one could also say that the variable income has a correlation of 0.65 with Factor 1. This would be considered a strong association for a factor analysis in most research fields.<\/p>\n\n\n\n

Two other variables, education and occupation, are also associated with Factor 1. Based on the variables loading highly onto Factor 1, we could call it \u201cIndividual socioeconomic status.\u201d<\/p>\n\n\n\n

House value, number of public parks, and number of violent crimes per year, however, have high factor loadings on the other factor, Factor 2. They seem to indicate the overall wealth within the neighborhood, so we may want to call Factor 2 \u201cNeighborhood socioeconomic status.\u201d The variable house value also is marginally important in Factor 1 (loading = 0.38). This makes sense, since the value of a person\u2019s house should be associated with his or her income.<\/p>\n\n\n\n

Definition<\/strong><\/p>\n\n\n\n

Suppose we have a set of observable random variables,  \"Image with means  \"Image Suppose for some unknown constants \"Image and  \"Image unobserved random variables \"Image where  \"Image and  \"Image, where   \"Image, we have<\/p>\n\n\n\n

\"Image<\/figure><\/div>\n\n\n\n

Here, the are independently distributed error terms with zero mean and finite variance, which may not be the same for all \u00ed Let \"Image so that we have<\/p>\n\n\n\n

\"Image<\/figure><\/div>\n\n\n\n

In matrix terms, we have<\/p>\n\n\n\n

\"Image<\/figure><\/div>\n\n\n\n

If we have \u03b7 observations, then we will have the dimensions  \"Image and  \"Image. Each column of \"Image  and \"Image denote values for one particular observation, and matrix \"Image does not vary across observations.<\/p>\n\n\n\n

Also we will impose the following assumptions on \"Image <\/p>\n\n\n\n

  • \"Image and are independent.<\/li>
  • \"Image<\/li>
  • \"Image (to make sure that the factors are uncorrelated).<\/li><\/ul>\n\n\n\n
    \"Image<\/figure><\/div>\n\n\n\n

    Any solution of the above set of equations following the constraints for \"Image is defined as the factors, and \"Image as the loading matrix. Suppose \"Image. Then note that from the conditions just imposed on\"Image, we have      <\/p>\n\n\n\n

    or<\/p>\n\n\n\n

    \"Image<\/figure><\/div>\n\n\n\n

    or<\/p>\n\n\n\n

    \"Image<\/figure><\/div>\n\n\n\n

    For any orthogonal matrix \"Image if we set \"Image and \"Image,the criteria for being factors and factor loadings still hold. Hence a set of factors and factor loadings is identical only up to orthogonal transformation.<\/p>\n\n\n\n

    Example<\/strong><\/p>\n\n\n\n

    The following example is for expository purposes, and should not be taken as being realistic. Suppose a psychologist proposes a theory that there are two kinds of intelligence, “verbal intelligence” and “mathematical intelligence”, neither of which is directly observed. Evidence for the theory is sought in the examination scores from each of 10 different academic fields of 1000 students. If each student is chosen randomly from a large population, then each student’s 10 scores are random variables. The psychologist’s theory may say that for each of the 10 academic fields, the score averaged over the group of all students who share some common pair of values for verbal and mathematical “intelligences” is some constant times their level of verbal intelligence plus another constant times their level of mathematical intelligence, i.e., it is a combination of those two “factors”. The numbers for a particular subject, by which the two kinds of intelligence are multiplied to obtain the expected score, are posited by the theory to be the same for all intelligence level pairs, and are called “factor loadings” for this subject. For example, the theory may hold that the average student’s aptitude in the field of taxonomy is<\/p>\n\n\n\n

    {10 \u00d7 the student’s verbal intelligence} + {6 \u00d7 the student’s mathematical intelligence}.<\/p>\n\n\n\n

    The numbers 10 and 6 are the factor loadings associated with taxonomy. Other academic subjects may have different factor loadings. Two students having identical degrees of verbal intelligence and identical degrees of mathematical intelligence may have different aptitudes in taxonomy because individual aptitudes differ from average aptitudes. That difference is called the “error” \u2014 a statistical term that means the amount by which an individual differs from what is average for his or her levels of intelligence (see errors and residuals in statistics).<\/p>\n\n\n\n

    The observable data that go into factor analysis would be 10 scores of each of the 1000 students, a total of 10,000 numbers. The factor loadings and levels of the two kinds of intelligence of each student must be inferred from the data.<\/p>\n","protected":false},"excerpt":{"rendered":"

    Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. For example, it is possible that variations in four observed variables mainly reflect the variations in two unobserved variables. Factor analysis searches for such joint variations in response to…<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-109036","page","type-page","status-publish","hentry"],"acf":[],"yoast_head":"\nFactor Analysis - Tutorial<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Factor Analysis - Tutorial\" \/>\n<meta property=\"og:description\" content=\"Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. For example, it is possible that variations in four observed variables mainly reflect the variations in two unobserved variables. Factor analysis searches for such joint variations in response to...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/\" \/>\n<meta property=\"og:site_name\" content=\"Tutorial\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/vskills.in\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-12T09:00:23+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-88-2.jpg\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/\",\"url\":\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/\",\"name\":\"Factor Analysis - Tutorial\",\"isPartOf\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-88-2.jpg\",\"datePublished\":\"2021-02-17T11:50:55+00:00\",\"dateModified\":\"2024-04-12T09:00:23+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#primaryimage\",\"url\":\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-88-2.jpg\",\"contentUrl\":\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-88-2.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.vskills.in\/certification\/tutorial\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Factor Analysis\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#website\",\"url\":\"https:\/\/www.vskills.in\/certification\/tutorial\/\",\"name\":\"Tutorial\",\"description\":\"Vskills - A initiative in elearning and certification\",\"publisher\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.vskills.in\/certification\/tutorial\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#organization\",\"name\":\"Vskills\",\"url\":\"https:\/\/www.vskills.in\/certification\/tutorial\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.vskills.in\/certification\/tutorial\/wp-content\/uploads\/2017\/07\/vskills-min-logo.jpg\",\"contentUrl\":\"https:\/\/www.vskills.in\/certification\/tutorial\/wp-content\/uploads\/2017\/07\/vskills-min-logo.jpg\",\"width\":73,\"height\":55,\"caption\":\"Vskills\"},\"image\":{\"@id\":\"https:\/\/www.vskills.in\/certification\/tutorial\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/vskills.in\/\",\"https:\/\/x.com\/vskills_in\",\"https:\/\/www.linkedin.com\/company-beta\/1371554\/\",\"https:\/\/www.youtube.com\/channel\/UCMWnscxPwRF_PqXo9B7q_Tw\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Factor Analysis - Tutorial","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/","og_locale":"en_US","og_type":"article","og_title":"Factor Analysis - Tutorial","og_description":"Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. For example, it is possible that variations in four observed variables mainly reflect the variations in two unobserved variables. Factor analysis searches for such joint variations in response to...","og_url":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/","og_site_name":"Tutorial","article_publisher":"https:\/\/www.facebook.com\/vskills.in\/","article_modified_time":"2024-04-12T09:00:23+00:00","og_image":[{"url":"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-88-2.jpg","type":"","width":"","height":""}],"twitter_misc":{"Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/","url":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/","name":"Factor Analysis - Tutorial","isPartOf":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#primaryimage"},"image":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#primaryimage"},"thumbnailUrl":"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-88-2.jpg","datePublished":"2021-02-17T11:50:55+00:00","dateModified":"2024-04-12T09:00:23+00:00","breadcrumb":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#primaryimage","url":"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-88-2.jpg","contentUrl":"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-88-2.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/factor-analysis\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.vskills.in\/certification\/tutorial\/"},{"@type":"ListItem","position":2,"name":"Factor Analysis"}]},{"@type":"WebSite","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#website","url":"https:\/\/www.vskills.in\/certification\/tutorial\/","name":"Tutorial","description":"Vskills - A initiative in elearning and certification","publisher":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.vskills.in\/certification\/tutorial\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#organization","name":"Vskills","url":"https:\/\/www.vskills.in\/certification\/tutorial\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#\/schema\/logo\/image\/","url":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-content\/uploads\/2017\/07\/vskills-min-logo.jpg","contentUrl":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-content\/uploads\/2017\/07\/vskills-min-logo.jpg","width":73,"height":55,"caption":"Vskills"},"image":{"@id":"https:\/\/www.vskills.in\/certification\/tutorial\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/vskills.in\/","https:\/\/x.com\/vskills_in","https:\/\/www.linkedin.com\/company-beta\/1371554\/","https:\/\/www.youtube.com\/channel\/UCMWnscxPwRF_PqXo9B7q_Tw"]}]}},"_links":{"self":[{"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/pages\/109036","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/comments?post=109036"}],"version-history":[{"count":1,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/pages\/109036\/revisions"}],"predecessor-version":[{"id":109037,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/pages\/109036\/revisions\/109037"}],"wp:attachment":[{"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/media?parent=109036"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/categories?post=109036"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.vskills.in\/certification\/tutorial\/wp-json\/wp\/v2\/tags?post=109036"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}