Example<\/strong><\/p>\n\n\n\n Consider the following list of seventeenth- and eighteenth-century writers.<\/p>\n\n\n\n Imagine that we are given two fragments of text written by one or two of these writers, and charged with identifying the true author(s). To make things interesting, imagine also that the only information we are given about an unidentified fragment of text is the frequency with which certain letters appear in it. Accordingly, I have taken three distinct samples of about 1000 characters each from the writings of each these authors, and added up the number of times each of the following characters appears in each of the samples (restricting ourselves to less than the complete alphabet prevents the tables in the rest of the discussion from becoming unwieldy; the characters chosen happen to occur with middling frequency in all the texts as a whole).<\/p>\n\n\n\n This is the cross tabulation.<\/p>\n\n\n\n Is it possible to say with reasonable certainty that the distribution of letters differs significantly from sample to sample (i.e., from row to row in the cross tab)? The usual means of answering such questions is Pearson\u2019s Assume that the cross tab under examination is described formally by the Next, define row and column totals:<\/p>\n\n\n\n The Here If rows and columns really are independent (i.e., \u201cunder the null hypothesis\u201d), Thus there is (almost) no probability under the null hypothesis of observing a statistic as large as the one actually observed, and indeed only a 1% probability of seeing a value about half as large. According to the Unfortunately, the For the purposes of correspondence analysis, the differences between the distributions of letters in the text samples\u2014which you will recall are given in the rows of the cross tab\u2014are measured by so-called This computes the Certain characteristics of the samples can be detected in the table above. For example, it appears that the Mark Twain (MT) texts form a relatively isolated group, in that the distances from the MT samples to each other are considerably smaller than from the MT samples to those of other authors. By itself, however, the table does little to make apparent the overall pattern of the distances\u2014something done in the next section. Before that, however, here is a little more on the nature of As their name suggests, From equation (5), since the row total for the centroid is 1 (by the definition of P<\/em>), the Now Drawing an analogy with the physical concept of angular inertia, correspondence analysis defines the inertia<\/em> of a row as the product of the row total (which is referred to as the row\u2019s mass<\/em>) and the square of its distance to the centroid, The total inertia of a table is used to assess the quality of its graphical representation in correspondence analysis.<\/p>\n\n\n\n Calculating Row Scores<\/strong><\/p>\n\n\n\n Correspondence analysis provides a means of representing a table of Next, we compute the singular value decomposition<\/em> of \u03a9 which is to say that we find orthogonal matrices The scores<\/em> of the rows\u2014whose interpretation we discuss later\u2014are given by the expression<\/p>\n\n\n\n Here The scores of the rows in our sample cross tab are computed. The row scores may be thought of as the coordinates of points in a high-dimensional space (14-dimensional, as it turns out in this case).<\/p>\n\n\n\n Plotting Rows<\/strong><\/p>\n\n\n\n Although the row scores faithfully reproduce the The following displays each row\u2019s (abbreviated) label at the position given by its coordinates and returns a key to the abbreviations.<\/p>\n\n\n\n Figure 1. <\/strong>Row plot for text samples.<\/p>\n\n\n\n The plot gives a much clearer picture of the way in which the letters are distributed across the text samples. For example, it is quite evident that\u2014as we concluded from the original cross tab\u2014the Mark Twain samples differ significantly as a group from those of the other writers. The text samples of Darwin and Hobbes also appear to be sui generis<\/em>, though the Descartes and Shelly samples appear less distinct. The plot suggests that it may be possible, therefore, to distinguish between the works of at least some of the authors using correspondence analysis of their letter distributions.<\/p>\n\n\n\n Diagnostics<\/strong><\/p>\n\n\n\n Since it uses only the first two components of the row scores, the plot above only approximates the true configuration of the rows in the cross tab. Before using it to make firm inferences, we might try to gauge the quality of the representation it provides. One indicator is derived from the inertia of the rows defined in (9). Recall that It may be shown that for any contingency matrix, the procedure of the previous section always places the centroid at the origin of the plot. Therefore, since Euclidean distances on the plot are supposed to approximate Thus our two-dimensional plot captures about 56% of the total inertia of the table rows. While this seems hardly an impressive fraction, Murtagh points out that ratios like this are not uncommon in correspondence analysis, and do not necessarily point to a bad representation. Nonetheless, we might want to exercise a modicum of caution before drawing categorical conclusions from our analysis.<\/p>\n\n\n\n As an aside, it turns out that the total inertia of the contingency matrix \u03a1\u2014which was calculated \u201clonghand\u201d in (9)\u2014is equal to the sum of the squares of the diagonal elements of the matrix in (11). The latter comprise the singular values<\/em> of the matrix \u03a9. Furthermore, the inertia retained in the two-dimensional plot is simply the sum of the squares of the first two singular values in \u2227.<\/p>\n\n\n\n Plotting Columns<\/strong><\/p>\n\n\n\n We have seen how correspondence analysis can be used to derive a visual representation of the relationships between the rows of a contingency matrix. We can also use correspondence analysis to illustrate the relationship between the rows and the columns of a correspondence matrix\u2014between the texts and letters in our example. Since our primary concern is with the text samples, the rows of the cross tab, it might seem a digression to look at the cross tab columns (the characters appearing in the texts), but we will see in the next section that the geometry of the columns is central to the identification of the mystery texts.<\/p>\n\n\n\n As with the rows, we begin by deriving scores for the columns from the singular value decomposition in (11). With reference to (11), the matrix C, whose rows are the column scores, is calculated as follows:<\/p>\n\n\n\n Where<\/p>\n\n\n\n As before, the two-dimensional column coordinates are simply the first two components of the scores.<\/p>\n\n\n\n We can display both columns and rows on the same plot with a slight elaboration of the method used to plot the rows alone. The column coordinates are scaled so that the column points occupy roughly the same region of the plot as the row points.<\/p>\n\n\n\n Figure 2. <\/strong>Row and column plot for text samples.<\/p>\n\n\n\n Interpreting the relationships between rows and columns from a plot such as this is not as straightforward as it was for the previous plot with the rows only. For example, it is not true in general that the closer a column appears to a row, the greater the prevalence of the corresponding letter in the corresponding text sample. To show how such relationships are actually represented, consider the text sample \u201cMT2\u2033 (a row) and characters \u201cP\u201d and \u201cY\u201d (columns).<\/p>\n\n\n\n Possibly the simplest way to determine the relationship between a text sample and a character is to draw lines from their corresponding points in the plot to the origin. If the angle between the two lines is acute<\/em>, then the character occurs more often in the sample than it does on average in the texts as a whole. Conversely, if the angle is obtuse<\/em>, the character occurs less often than overall. The following draws the appropriate lines for our chosen text sample and characters; it appears the character \u201cY\u201d occurs more often than average in \u201cMT2\u2033, while \u201cP\u201d occurs less often.<\/p>\n\n\n\n Figure 3. <\/strong>Simple analysis of row\/column plot.<\/p>\n\n\n\n Unfortunately, the method described above only tells us if a character appears more or less often than average in a text sample, not whether one character appears more often than another in a sample. In particular, an angle that is more acute does not signify a character that is more prevalent in a text.<\/p>\n\n\n\n A rather more complicated method that does illustrate the relative frequencies of characters in a text sample entails first drawing a line on the plot through the origin and the point corresponding to the text sample in question. Perpendiculars to this line are dropped from each character\u2019s position on the plot. The following draws such a construction for the selected text sample \u201cMT2\u2033.<\/p>\n\n\n\n Figure 4. <\/strong>Comprehensive analysis of row\/column plot.<\/p>\n\n\n\n The relative frequencies of the characters in the text sample can be read off by traversing the line through the text sample (colored blue and green on the plot above), looking at the positions at which the perpendiculars from the characters intersect it. A character with an intersection on the green line segment (i.e., on the same side of the origin as the text sample) occurs more often in the sample than the average in the texts overall, whereas one on the blue line segment (on the other side of the origin) occurs less frequently than the average. In addition, the further from the origin on the green line segment such an intersection occurs, the greater the frequency of the character in the sample. Conversely, the further out on the blue segment an intersection falls, the less frequent the character in the sample.<\/p>\n\n\n\n So from the plot above, it appears that the character \u201cW\u201d occurs most often in the sample text, and that characters \u201cL\u201d, \u201cD\u201d, \u201cY\u201d, \u201cG\u201d, \u201cU\u201d, \u201cB\u201d, \u201cS\u201d, \u201cI\u201d, \u201cN\u201d, \u201cH\u201d, \u201cC\u201d, \u201cM\u201d, \u201cP\u201d, \u201cF\u201d, and \u201cR\u201d occur successively less often; characters \u201cW\u201d through \u201cB\u201d in the ranking appear more often than average, while \u201cS\u201d through \u201cR\u201d appear less often than average.<\/p>\n\n\n\n Supplementary Points: Identifying the Mystery Texts<\/strong><\/p>\n\n\n\n Finally, we return to the problem we faced at the outset: identifying the author or authors of the unidentified text fragments. We have seen how the application of simple correspondence analysis to the text samples allows us to view them graphically in terms of their letter distributions. In Section 5 we saw that it was generally possible to distinguish the authors of the text samples based upon the locations of the corresponding row points\u2014with a few exceptions, samples of work by the same writer tended to occupy the same area of the plot. One might logically surmise that if we were to plot the mystery texts on the same correspondence plot as the samples, we would be able to determine their authorship by looking at the authors of the nearest samples. To begin, we need to calculate an additional cross tab containing the distribution of the selected characters in the mystery texts.<\/p>\n\n\n\n We could proceed by simply appending these frequencies as new rows to the original text samples cross tab given in Section 1 and recalculating the scores and coordinates for all the rows (that is, both the original samples and the mystery texts) in the resulting table. In principle, however, it is possible that the unidentified texts overlap one or more of the text samples, and if this were the case, appending the new rows to the cross tab would distort the analysis by \u201cdouble-counting\u201d some of the samples.<\/p>\n\n\n\n A more satisfactory approach derives from the fact that the row scores computed in Section 4 are actually weighted sums of the column scores calculated in Section 6. In matrix terms, recalling that \u03a1 is the correspondence matrix and C is the matrix of column scores, it can be shown that<\/p>\n\n\n\n Where<\/p>\n\n\n\n If we replace the original correspondence matrix in (17) with a new correspondence matrix formed from the cross tab of the unidentified texts, we derive a set of row scores for the unidentified texts according to the transformation determined by the text samples only (since they alone produced the row scores ), eliminating the risk of double-counting. Treated in this way, the unidentified texts comprise supplementary points<\/em> in the terminology of correspondence analysis.<\/p>\n\n\n\n The following calculates row scores for the mystery texts as supplementary points (straightforward algebra vindicates the direct use of the new cross tab without the need to derive a new correspondence matrix).<\/p>\n\n\n\n Lastly, as with the rows and columns, we take the first two elements of the scores above to produce the coordinates of the supplementary points. In the following, they are displayed on the same plot as the original rows.<\/p>\n\n\n\n Figure 5. <\/strong>Plot of the mystery texts as supplementary points.<\/p>\n\n\n\n All points on this plot represent texts, or rows, and distances between points can be interpreted directly as degrees of similarity, just as with the row plot in Section 5. On this basis, judging by their closeness to the authors\u2019 other works, it appears that mystery texts 1 and 2 belong to Mark Twain and Thomas Hobbes respectively. While the manifest isolation of the Mark Twain texts on the plot leaves little doubt as to the provenance of the first unidentified text, the author of the second is a little less clearly defined\u2014particularly given the middling diagnostic ratio calculated in Section 5. Nonetheless, I am sure you will agree that considering the rather scant literary information on which the analysis was based (amounting to no more than a table of letter frequencies), the results are encouraging.<\/p>\n","protected":false},"excerpt":{"rendered":" Correspondence analysis (CA) is a multivariate statistical technique proposed by Hirschfeld and later developed by Jean-Paul Benz\u00e9cri. It is conceptually similar to principal component analysis, but applies to categorical rather than continuous data. In a similar manner to principal component analysis, it provides a means of displaying or summarising a set of data in two-dimensional…<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-109040","page","type-page","status-publish","hentry"],"acf":[],"yoast_head":"\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-128-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-129-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-130-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-131-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-132-2.jpg\")
Calculations<\/strong><\/p>\n\n\n\ntest for independence; it tests whether a cross tab deviates significantly from one in which rows and columns are independent. In our case, independence would imply that the letters occur with the same frequency in all of the text samples.<\/p>\n\n\n\n
![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-133-2.jpg\")
matrix ![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-134-2.jpg\")
. We derive the correspondence matrix p from F by dividing its entries by their grand total.<\/em> ![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-135-2.jpg\")
(1)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-136-2.jpg\")
(2)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-137-2.jpg\")
statistic, , is calculated:<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-138-2.jpg\")
(3)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-173-1.jpg\")
is an estimate of an entry\u2019s value assuming independence:<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-139-2.jpg\")
(4)<\/p>\n\n\n\n should follow a distribution with ![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-174-1.jpg\")
degrees of freedom.<\/p>\n\n\n\n test, therefore, there is a statistically significant difference in the distribution of letters across the samples.<\/p>\n\n\n\n test by itself does not provide a solution to the problem of distinguishing the works of the different authors. Though it establishes that the distribution of letters differs significantly from one sample to another, it does not tell us whether the samples of one author differ from those of other authors more than they differ from each other, nor does it allow us to characterize the authors in terms of the distribution of letters in their works. Answers to these questions are provided by correspondence analysis.<\/p>\n\n\n\n Distances<\/strong><\/p>\n\n\n\n distances<\/em>, which are weighted Euclidean distances between normalized rows (calculated by dividing row entries by their respective row total), with weights inversely proportional to the square roots of the column totals. In symbols, the distance between row and row is given by the expression:<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-140-2.jpg\")
(5)<\/p>\n\n\n\n distances between the text samples using the correspondence matrix and displays them in a reasonably compact table (after scaling up by 100 and rounding).<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-141-2.jpg\")
<\/figure><\/div>\n\n\n\n distances.<\/p>\n\n\n\n distances are closely related to the statistic of the previous section. To show how they are related, consider the \u201caverage\u201d row\u2014termed the centroid<\/em> or barycenter<\/em> in correspondence analysis\u2014whose entries are simply the column totals<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-142-2.jpg\")
(6)<\/p>\n\n\n\n distance of row i<\/em> to the centroid is<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-143-2.jpg\")
(7)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-145-2.jpg\")
with as defined in (4)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-144-2.jpg\")
(8)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-146-2.jpg\")
Comparing the expression for ![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-1-7.jpg\")
in (5) with definition of the statistic in (3), it follows that the total inertia of all the rows in a contingency matrix is equal to the statistic divided by ![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-2-7.jpg\")
a quantity known as Pearson\u2019s mean-square contingency<\/em>, denoted \u00f82 :<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-148-2.jpg\")
(9)<\/p>\n\n\n\ndistances in a graphical form, with rows represented by points, so that the distances between points approximate the distances between the rows they represent. To compute such a representation, we begin with a matrix of standardized residuals<\/em>, which are the square roots of the terms comprising the statistic in (3)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-149-2.jpg\")
(10)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-3-8.jpg\")
, together with a diagonal matrix \u2227, such that (with the transpose of matrix \u039c denoted ![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-150-2.jpg\")
, and writing \u0399 for the identity matrix)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-151-2.jpg\")
(11)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-152-2.jpg\")
(12)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-4-8.jpg\")
is the diagonal matrix comprising the reciprocals of the square roots of the row totals.<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-153-2.jpg\")
(13)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-154-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-155-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-156-2.jpg\")
<\/figure><\/div>\n\n\n\n distances between rows in the original table, as coordinates their dimensionality is far too high for them to be presented graphically. Thanks to the properties of the singular value decomposition, however, taking just the first two components of each row\u2019s score usually produces a reasonable approximation to the distances, and yields coordinates that can be placed on a two-dimensional plot. (Below we have labeled the components \u201cX\u201d and \u201cY\u201d to highlight their role as 2D coordinates.)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-157-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-158-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-159-2.jpg\")
, the total inertia of the rows, is calculated from the row totals and the distances of the rows to the centroid<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-160-2.jpg\")
<\/figure><\/div>\n\n\n\n distances, replacing each distance ![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-5-8.jpg\")
in the right-hand side of (14) by the distance of the corresponding row point to origin should yield an approximation to . The following derives this quantity and computes its ratio to the true value of .<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-161-2.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-162-1.jpg\")
(16)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-163-1.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-164-1.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-165-1.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-166-1.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-167-1.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-168-1.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-169-1.jpg\")
(17)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-170-1.jpg\")
(18)<\/p>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-171-1.jpg\")
<\/figure><\/div>\n\n\n\n![\"Image](\"http:\/\/www.vskills.in\/lms\/wp-content\/uploads\/2016\/07\/Image-172-1.jpg\")
<\/figure><\/div>\n\n\n\n