Wednesday, July 17, 2013

Technical innovation and spatial ability




A new paper from David Lubinski and collaborators looks at spatial ability measured at age 13 to see whether it adds predictive power to (SAT) Math and Verbal ability scores. The blobs in the figure above (click for larger version) represent subgroups of individuals who have published peer reviewed work in STEM, Humanities or Biomedical research, or (separately) have been awarded a patent. Units in the figure are SDs within the SMPY population.
Creativity and Technical Innovation: Spatial Ability’s Unique Role
DOI: 10.1177/0956797613478615

In the late 1970s, 563 intellectually talented 13-year-olds (identified by the SAT as in the top 0.5% of ability) were assessed on spatial ability. More than 30 years later, the present study evaluated whether spatial ability provided incremental validity (beyond the SAT’s mathematical and verbal reasoning subtests) for differentially predicting which of these individuals had patents and three classes of refereed publications. A two-step discriminant-function analysis revealed that the SAT subtests jointly accounted for 10.8% of the variance among these outcomes (p < .01); when spatial ability was added, an additional 7.6% was accounted for—a statistically significant increase (p < .01). The findings indicate that spatial ability has a unique role in the development of creativity, beyond the roles played by the abilities traditionally measured in educational selection, counseling, and industrial-organizational psychology. Spatial ability plays a key and unique role in structuring many important psychological phenomena and should be examined more broadly across the applied and basic psychological sciences.
Note that SAT composite accounted for 10 percent of variance in research success even within this already gifted subpopulation. This non-zero result, despite the restriction of range, contradicts the Gladwellian claim that IQ above 120 does not provide additional returns. In fact, the higher the IQ score above the 99.5 percentile cutoff for this group, the greater the likelihood that an individual has been awarded a patent or has published a research paper.

32 comments:

  1. Since psychometricians have always apparently agreed that there seem to be three principal subcomponents of g, I've sometimes been a little curious about why the SAT only tests for two of them.


    My guess is that those were the two subcomponents considered most important by the particular academics who originally developed the SAT...

    ReplyDelete
  2. David Coughlin7:40 AM

    This is interesting: psychometricians have always apparently agreed that there seem to be three principal subcomponents. I have always assumed that spatial and logical reasoning get lumped under the name 'math'.

    ReplyDelete
  3. Stevie Mac6:40 AM

    I think physics is a very visualization demanding subject. You have to imagine things at all different levels and picture them in order to think about them. That's exactly what I can't do very much at all now. Metaphysics, one of my favourtie subjects, is pretty much out too for the most part.

    I don't know if my little story will tell you anything that isn't obvious but its interesting that this can happen, how big the implications are, and how other parts of intelligence can remain in tact. Incidentally, I got an MRI scan and it showed no damage, though apparently that only looks at the big structures and would not necessarily pick up changes to more small scale, intricate wiring. It always felt to me like a kind of brain damage. It is certainly a kind of mind damage.

    ReplyDelete
  4. BlackRoseML4:27 PM

    Stevie, take this test.

    http://psych.io/spatial/


    I want to know how you would do. Don't let it affect your self-esteem. You may be able to get 9/9 before. Maybe you are now a 5/9/. Please don't randomly guess, but educated guesses are acceptable.

    ReplyDelete
  5. David Coughlin6:28 PM

    I'm a sucker for online tests. I'm glad that wasn't timed. Problem 6 was just too much for me without a pencil and paper.

    ReplyDelete
  6. BlackRoseML6:34 PM

    I got #6 correctly without pencil and paper, but it took about five minutes.

    ReplyDelete
  7. David Coughlin6:54 PM

    I didn't use pencil and paper [that would be cheating, right?]. Everything else seemed pretty straightforward.

    ReplyDelete
  8. Richard Seiter7:07 PM

    It would be good to indicate whether or not pencil and paper should be used. I wasn't sure, but using them seemed to defeat the purpose of the test so I chose not to. #6 was tough. 5 minutes sounds about right for me. Can you give any idea of the distribution of results you see (beyond the mean given) and how they align with other tests? (do you have any papers out?) Have you done any studies of HOW people solve the problems? I feel like I was using skills other than pure spatial ability. My primary solution approach was to identify patterns and follow them around a sequence of four squares (for #6 and to a lesser degree #9 the spatial skills were needed, not so much for the others). I felt like working memory capacity was limiting me as much as spatial skill on #6. It would be interesting to hear how a spatial thinker like Stevie approaches solving these problems.

    ReplyDelete
  9. BlackRoseML7:30 PM

    Me too...

    I posted this on FuturePundit:

    I always considered my spatial ability to be crap; I suppose I have high enough g to do well on an untimed, supposedly "difficult" spatial test, even though I consider it to be my weakest aptitude, thus it is not "crap" in an absolute sense or when compared against the general population.I just noticed things whether the faces are clockwise, if the symbols are upright, or if the opposite faces are proper (for instance, all correct figures would have a heart and a cross opposite, but the incorrect one, they are adjacent). I am like Khan Noonien Singh, mostly exhibiting two dimensional thinking... I didn't visualize the cubes much and rotate the figures in my head though,
    but used heuristics described above to logically figure out the
    incorrect one.

    ReplyDelete
  10. Richard Seiter7:32 PM

    More on this test from a Hacker News posting by one of the test's developers:
    https://news.ycombinator.com/item?id=3186571

    Here is one he noted as a harder question: http://psych.io/spatial/hard.jpg

    ReplyDelete
  11. Richard Seiter7:46 PM

    Thought folks here might be interested in this paper:
    Can spatial training improve long-term outcomes for gifted STEM undergraduates?
    http://groups.psych.northwestern.edu/uttal/people/documents/MillerHalpern-inpressfinal-Spatialtraining.pdf

    ReplyDelete
  12. BlackRoseML8:44 PM

    I think i got the answer to that...

    ReplyDelete
  13. David Coughlin8:53 PM

    I will add that I just had to cut the cubes out for number 6 and see it. I actually thought about the specific feature that distinguishes it but I just couldn't see the whole cube in my head to visualize it. You really have to see behind the cube to get it.

    ReplyDelete
  14. dxie489:02 PM

    Interestingly one of the most important technology for current digital communication is the frequency-hopping spread-spectrum technology (used in such things as Bluetooth, COFDM used in Wi-Fi network connections, and CDMA used in some cordless and wireless telephones) was invented by a duo of actor and musical composer with artistic and tonal ability,

    http://en.wikipedia.org/wiki/Hedy_Lamar

    Lamar was cast alongside Lana Turner and Judy Garland in Ziegfeld Girl.

    ReplyDelete
  15. Richard Seiter9:04 PM

    David, try looking for the pattern circle, up arrow, square, square in a row of four (like in 6C) and see if you can find the different one that way. That's how I did it. I found it much easier to visualize a "ring" of four than I did the whole cube. The two squares in a row limit the search space.

    ReplyDelete
  16. BlackRoseML9:30 PM

    I meant to say that i could visualize folding the cube in my head, but i could not rotate the folded cube mentally. i used other ways to "see" what was behind the cube.

    ReplyDelete
  17. David Coughlin10:07 PM

    I vis'ed the ring of four pretty easily. Since this is way down the comments, I don't mind posting a spoiler. In A, when you are looking at a circle, and the arrows are 'flowing' around it, they are flowing clockwise. In the others, the arrows are flowing counterclockwise. I could see the vertex-on-perspective pretty easily with three faces visible, and I could see the arrow-arrow-square-circle [if you construe that the arrows are pointing at the 'next' face] sequence pretty easily. I couldn't synthesize in my head the direction of the arrows relative to the non-AASC-sequence faces. I even knew that the distinguishing feature had to be determined by the arrows direction.

    ReplyDelete
  18. tractal11:39 PM

    LOL at that abstract. WE MUST DO MORE.

    ReplyDelete
  19. BlackRoseML12:35 AM

    I would guess that's Harvey Mudd

    ReplyDelete
  20. BlackRoseML4:28 AM

    I was a STEM major (cell and molecular biology) and I do pursue STEM topics in my own free time if a given topic interests me. I said this before:

    STEM does not require a high-level of specific verbal ability (that is verbal ability that is independent of general intelligence), and above a certain
    threshold of “g” (probably 1 SD) quantitative ability has more utility when studying any STEM discipline. Those who are below that moderate threshold of "g" probably lack the reading comprehension ability to understand the contents of scientific literature (and also, in the context of this discussion, the spatial ability to visualize the description of whatever phenomena the scientific literature is describing). Most scientific literature is not verbally demanding though, since it just requires familiarity with the lexicon and concepts, and that can be obtained through a combination of memorization and general intelligence. For example, I read parts of The City of God before I converted (it did not cause me to convert but it perhaps made Christianity more intellectually respectable to me as an agnostic), and I could say that comprehension of the translations require high general verbal intelligence (and a rudimentary knowledge of Christian history and its tenets) in order to comprehend Augustine’s arguments, while the opacity of scientific literature comes from its liberal references to content-specific knowledge and technical vocabulary that require one to be familiar with scientific principles and the body of knowledge that pertains the field in question.

    I do therefore believe that spatial ability is more relevant to STEM achievement than verbal ability though. There was a long discussion here more than a year ago where some of the participants argued that it is not "g" but special cognitive abilities (including spatial) that are the necessary aptitude for genius level performance and achievement in an intellectual domain. As one who subscribes to the Spearman's Law of Diminishing returns, it seems to me that specific abilities become more important for higher level performance since the relevant aptitude needed in an given intellectual domain becomes less correlated with "g" as "g" increases; and for high "g" individuals, their cognitive profiles are often varied and diverse and, therefore, one cannot predict high-level performance in specific domains simply by knowing one's "g". The abstract provides evidence for this by stating the spatial ability offers unique predictive power of STEM achievement even when other aspect of "g" such as quantitative and verbal ability are controlled. For those lacking "g", one can conflate one's proficient for a specific cognitive ability with his/her level of "g".

    Due to deficiencies in "g" and consequentially, specific facets of "g" such as spatial, mathematical, and to a lesser extent verbal ability, it would seem impossible to increase the STEM achievement in the general population. For instance this paper, reports that among the STEM Ph.D. holders of the Project Talent cohort, less than 10% were below the top quartile of spatial ability in adolescence. I would speculate that those individuals have lopsided cognitive ability profiles were they either had high quantitative or verbal ability that was detected through standardized testing, but if general cognitive ability limits spatial ability in the lower levels of "g", it would seem that those below the top-quartile usually have their other cognitive abilities in that range too.

    ReplyDelete
  21. Stevie Mac9:46 AM

    I got 7 out of 9. I never answered number 7- could not discern any difference at all! I'm not sure if this is more visualisation ability or logic. It definitely involved mental folding, while relying on looking at the picture to do that, but I was very much engaging logically with it, picking out single rules then examining each one til I found an odd one out. I didn't time it but it took a lot longer than 5 minutes. Maybe around 20 or 30. I couldn't say. I kind of enjoyed it. I don't come through something like this unscathed though. There's some recovery time. My mind feels hyper aware of angles and spatial relations now like my mind can't let go of focusing on them. For example, I'm hyper aware than the keyboard is angled differently to the screen and that feels weird. It has twisted my mind up a bit.

    ReplyDelete
  22. Stevie Mac9:50 AM

    I'm impressed you did it in 5 minutes.

    ReplyDelete
  23. Jason Ko5:52 PM

    The whole test took me about 35-40mins (I really didn't understand how the scores could be standardized - well they don't say they can be - without a time limit). I scored 9 of 9, but again by taking as much time as needed for each question. Is there some kind of ceiling deviation score this test can measure? What place does it have amongst other psychometric tests in use?

    ReplyDelete
  24. Cornelius1:43 PM

    Based on these results - http://infoproc.blogspot.com/2008/07/annals-of-psychometry-iqs-of-eminent.html - I expect spatial ability to be the least important component of intelligence for achieving eminence in scientific research.

    ReplyDelete
  25. Richard Seiter10:09 AM

    What about the comment "(2) S scores tend to decrease with age (correlation .4). Peak (younger) performance would have been higher."? Roe goes on to comment (page 167 in my copy): "If these men had been tested 20 years earlier they might have scored as much higher on this test as they did on the other." and "The theoretical physicists are the youngest group, so this may in part account for their higher average on this test, although only in part." and goes on to speculate about a possible relation of spatial ability with work in physics and note that science students tend to have higher scores on non-verbal tests than arts students.


    You may be right, but I'm not sure how well your assertion is supported by Roe's work.


    Also germane to this thread are some observations Roe makes on page 170 about a .33 correlation between spatial and verbal scores and positing as a reason the observation that some people answer the spatial problems "just by looking" (common among those who do extremely well), but others do fairly well by going through a fairly intricate process of reasoning (with these people causing the verbal correlation).

    ReplyDelete
  26. Cornelius11:49 AM

    Do you know a good reference for the severity of that decrease? 1.5 s.d. below the average of Q+V seems quite severe unless the S score was relatively low to start with.

    ReplyDelete
  27. Richard Seiter2:20 PM

    The reference was Anne Roe's book (page 167) which was the source of Steve's post. Unfortunately the book does not contain a further reference. Sadly it has no list of references and minimal footnotes. If someone were really interested, Anne Roe's papers (see http://infoproc.blogspot.com/2013/03/roes-scientists.html ) might have more information. I thought her judgement was credible because she had the data and was mathematically literate (coauthor of Quantitative Zoology).

    Wikipedia mentions the age effect http://en.wikipedia.org/wiki/Spatial_visualization_ability#Age_differences but does not provide a reference.

    This reference seems to indicate a lower correlation coefficient:
    http://faculty.virginia.edu/cogage/publications2/Pre%201995/Effects%20of%20Age%20and%20Naturally%20Occurring%20Experience%20on%20Spatial.PDF

    I took a look at Wechsler's book amzn.com/B000PRWUME but did not see anything definitive on spatial ability and age though the correlation coefficients for verbal and performance scores were about 0.25 different (page 139).

    This might also be of interest:
    http://www.edgj.org/index.php/EDGJ/article/viewFile/49/48

    Better references would be appreciated. I agree 1.5 SD seems like a big difference.

    ReplyDelete
  28. Brett Olsen4:57 PM

    By my analysis, the "hard" question is incorrectly posed: B & C are unambiguously identical, A is unambiguously different, and D is the mirror image of B & C. I'd treat chiral boxes as being different, particularly because one or two of the questions on the main exam require you to do so to find the correct answer.


    I had to resort to pencil and paper for this one, mostly to confirm my supposition that D is ambiguous. The main exam I did without, and managed a 9/9. Fun test!

    ReplyDelete
  29. Yan Shen6:19 PM

    Posted this a few times before...

    "There's no doubt that Terry Tao reasons almost incredibly well, mathematically, and learns mathematics and related subjects astonishingly fast. His performance in mathematics competitions in Australia and on the mathematical portion of the College Board Scholastic Aptitude Test (SAT-M) at age 8 is phenomenal. He was taking the 60-item 60-minute multiple-choice SAT-M for the first time. On it, only 1 percent of college-bound male 12th-graders in the United States score 750 or more (College Board, 1985). He scored 760. Only one other 8-year-old of whom I am aware has done as well. That boy, who lives in a suburb of Chicago, was taking the test for the fifth time! He managed to score 800 before becoming 10 years old. Terry was not retested on SAT-M at age 9, because that seemed unnecessary.

    Yet at age 8 years 10 months, when he took both the SAT-M and the SAT-Verbal, Terry scored only 290 on the latter. Just 9% of college-bound male 12th-graders score 290 or less on SAT-V; a chance score is about 230. The discrepancy between being 10 points above the minimum 99th percentile on M and at the 9th percentile on V represents a gap of about 3.7 standard deviations. Clearly, Terry did far better with the mathematical reasoning items (please see the Appendix for examples) than he did reading paragraphs and answering comprehension questions about them or figuring out antonyms, verbal analogies, or sentences with missing words.

    Was the "lowness" of the verbal score (excellent for one his age, of course) due to his lack of motivation on that part of the test and/or surprise at its content? A year later, while this altogether charming boy was spending four days at my home during early May of 1985, I administered another form of the SAT-V to him under the best possible conditions. His score rose to 380, which is the 31st percentile. That's a fine gain, but the M vs. V discrepancy was probably as great as before. Quite likely, on the SAT score scale his ability had risen appreciably above the 800 ceiling of SAT-M.

    How could Terry possibly learn mathematics and physical and computer sciences so well with only 290-380V development? We of the Study of Mathematically Precocious Youth (SMPY) at Johns Hopkins have discovered, chiefly by testing able 12-year-olds, that when the examinee's SAT-M score vastly exceeds his or her SAT-V score the youth is almost certain to score high on a difficult test of nonverbal reasoning ability such as the Advanced Form of the Raven Progressive Matrices, often higher than a high-M high-V examinee does. To test this out, on 6 May 1985 I administered to Terry the RPM-Advanced, an untimed test. He completed its 36 8-option items in about 45 minutes. Whereas the average British university student scores 21, Terry scored 32. He did not miss any of
    the last, most difficult, 4 items. Also, when told which 4 items he had not answered correctly, he was quickly able to find the correct response to each. Few of SMPY's ablest protégés, members of its "700-800 on SAT-M Before Age 13" group, could do as well."

    ReplyDelete
  30. Richard Seiter12:31 PM

    This link might have some ideas for investigating spatial ability: http://www.wired.com/wiredscience/2013/11/map-sense/

    The thing that caught my eye was: "Another map-related skill that varies from person to person involves mental rotation. ... People who are good at this tend to keep north at the top when they read a map," I tend to favor direction of travel/view up for map interpretation. I can generally do the mental rotation but find it quicker/easier to avoid it. I wonder if this correlates with my description in other comments of being able to do the spatial tests, but having it feel like what I am doing is more g related. What do others experience?



    On another note, does anyone else think the first map question is problematic? The RHS curb looks straight to me in the picture, but not in the map.

    ReplyDelete
  31. IICIE has launched the Certified Business Innovation Manager
    (CBIM) Certification. This certification is aimed at providing a global
    standard for Innovation training and ensuring that a specific set of
    methodologies are implemented in any Innovation eco-system. This
    certification will cover the basics of innovation, different tools and
    techniques employed during the process, highlight major success stories
    of innovation and describe how innovation frameworks can be developed at
    an organization.

    ReplyDelete
  32. Hi there, all the time i used to check webpage posts here early in the dawn, for the reason that i like to gain knowledge of more and more.

    ReplyDelete