Executive Summary | Impact of Social Sciences

Executive Summary

Executive Summary

Defining research impacts

1. A research impact is a recorded or otherwise auditable occasion of influence from academic research on another actor or organization.
a. Academic impacts from research are influences upon actors in academia or universities, e.g. as measured by citations in other academic authors’ work.
b. External impacts are influences on actors outside higher education, that is, in business, government or civil society, e.g. as measured by references in the trade press or in government documents, or by coverage in specialist ‘close to policy’ media, or by coverage in mass media.
2. A research impact is an occasion of influence and hence it is not the same thing as a change in outputs or activities as a result of that influence, still less a change in social outcomes. Changes in organizational outputs and social outcomes are always attributable to multiple forces and influences. Consequently, verified causal links from one author or piece of work to output changes or to social outcomes cannot realistically be made or measured in the current state of knowledge.
3. A research impact is also emphatically not a claim for a clear-cut social welfare gain (i.e. it is not causally linked to a social outcome that has been positively evaluated or validated as beneficial to society in some way).
4. However, secondary impacts from research can sometimes be traced at a much more aggregate level, and some macro-evaluations of the economic net benefits of university research are feasible. Improving our knowledge of primary impacts as occasions of influence is the best route to expanding what can be achieved here.

What shapes the citing of academic publications?

5. Citation rates are now appropriately used as a basis for tracking academic impacts across all disciplines. The barriers to their use in the socials sciences, humanities and applied STEM disciplines have all greatly reduced in recent years. Yet the shape of citation rates still vary widely across academic disciplines.
6. There are substantial differences in the general rate of citing across disciplines with more cites (including self-cites) being found in the sciences than the social sciences.
7. The type of output chosen affects citation rates e.g. on average a book will take longer to be referred to but will be cited for longer.
8. How academics balance their time across the six areas of responsibility will be another important factor in citation rates.

Knowing your strengths

9. In the past academics have had few available tools to track their citation rates. We suggest using a combination of the three best tools which are Harzing’s Publish or Perish, Google Scholar and Google Book Search, and the ISI Web of Knowledge.
Google Scholar Citations also is a new facility that may have extensive impact and take-up in academia in the next 2-3 years.
10. Having a distinctive author name is very helpful for academics’ work to be easily found amongst a global deluge of information – academics should choose an author name that will allow their citations to be easily retrieved in global-scale publishing markets.
11. Conventional citation-tracking systems like ISI WOK and Scopus have limited coverage in the social sciences and humanities, and an American-based geographical bias, as well as capturing relatively few citations in languages other than English. They are only just planning to cover books and do not cover working papers.
12. Internet-based systems like Harzing’s Publish or Perish, Google and Scirus cover a wider range of academic outputs and now provide more reliable analysis of how research is being cited – they are much more reliable and inclusive in the social sciences and humanities.

Key measures of academic influence

13. Simple indicators for judging citation rates – such as total number of publications, total number of citations, and an age-weighted citation rate do not accurately capture an academics’ citation success.
14. Calculating an academic’s h-score and g-score provides a more robust picture of how much an academic’s work is valued by her peers.
15. Across all disciplines in the social sciences, journal articles account for the majority of citations, reflecting the large numbers of published articles. Books account for 8 to 30 per cent of citations across different disciplines. Books may figure disproportionately amongst those well-cited entries that build h scores and the g index. Book chapters in edited books, however, are often hard to find and are poorly referenced.
16. Network analysis can help shed light on the difference in citation rates between ‘hub’ and ‘authority’ academics at different stages in their careers, which compares the number of inward and outward citations.  

Getting better cited

17. Academics who wish to improve the citation rate of their journal articles should ensure that title names are informative and memorable, and that their abstracts contain key ‘bottom line’ or ‘take-away points’. ‘Narrative’ titles that in some way cue findings have many advantages in modern academic communication.
18. Book authors should ensure that their titles and sub-titles are distinctive yet appear in general ‘Google Book’ searches around the given theme.
19. There are a number of schools of thoughts regarding self-citations. In general academics should aim to ensure their own self-citation rate is in line with those of other academics in the same discipline.
20. Co-authored outputs tend to generate more citations due to networking effects between authors in a given research team or lab, especially if the co-authors come from different universities or countries.

Patterns of external research impacts

21. Generating impact within single academic disciplines is a complex process encompassing not only ‘discovery’ but also integration, application, and professional renewal; each of which make significant demands on an academic’s time and contribute to impacts. E.g. some of the academic impacts with the widest reach and greatest significance arise from integration work, not just applications or discovery work, especially in the social sciences
22. Academic work is highly siloed into disciplines while societal problems are multi-dimensional. Bridging scholarship across disciplines, promoting integration at the university level, and engaging in academic and professional service are some key additional ways in which academics’ work can better reach and influence wider society.
23. The ‘impacts interface’ describes how in advanced societies intermediaries – such as consultancies, professional networks, think tanks, specialist media, and other organisations – aggregate, distil and re-package trends in academic research for clients and other actors in the private sector, government, and civil society.
24. Academics giving informal advice to businesses, along with lectures, networking, contract work, student placements, joint publications and consultancy are the most widely undertaken activities likely to generate external impacts.

Is there an impacts gap?

25. Government officials and businesses often complain of an ‘impact gap’ where academic research fails to fulfil its potential to influence wider societal development. (The wider issue of ‘outcome gaps’ is too difficult to track or discuss due to the multi-causal nature of social life and the weak existing evidence base about such issues).
26. If there is an impacts gap it could be attributed to:

• demand and supply mismatches;
• insufficient incentives problems;
• poor mutual understanding and communication;
• cultural mismatch problems; or
• weak social networks and social capital.

27. Solutions to effectively combat an impacts gap cannot be homogenous across all academic disciplines and sectors, but rather should be innovative and tailored to the demonstrated problem.

How researchers achieve external impacts

28. While different authors and schools of thoughts within disciplines will take a different view of what factors make a difference to an academic achieving external impacts, we hypothesize that the following seven variable are the most relevant:

• his or her academic credibility;
• dispositional and sub-field constraints that rule some academics out of influence;
• networking skills;
• personal communication capacity;
• external reputation;
• experience of applied and joined-up work; and
• track record of successful applied and joined-up work.

29. Analysis of our pilot sample of 120 academics shows that academics who are cited more in the academic literature in social sciences are also cited more in non-academic Google references from external actors.
30. Researchers responding to research councils and funding bodies tend to claim impact in a haphazard way. It is possible to see in more detail analysis a more robust link between outputs produced for a particular project and objectively moderated impact assessments.

How organizations achieve external impact

31. While academic departments, labs, and research groups produce a great deal of explicit knowledge, it is their collective ‘tacit knowledge,’ which is the most difficult to communicate to external audiences, and yet that tends to have the most impact.
32. The changing nature of commissioned academic work means that the time lag in achieving external impacts can be radically reduced, yet any external impact of non-commissioned work is likely to lag far beyond its academic impact.
33. It is important for both individual departments/ research labs, schools or faculties, and the University as a whole to systematically collect, access and arrange auditable data on external impacts; keeping in mind that some ‘naïve customers’ like funders, regulators, and other parts of their universities may insist on proof of ‘extended’ impacts
34. Making meaningful comparisons between universities’ and individual departments’ external impacts requires contextual understanding of how departments and universities generally perform in a given country and institutional environment.
35. Seeking to improve external impact should not mean sacrificing academic independence and integrity. Compiling a risk assessment for working with external actors or funders is one way to mitigate the politicization of one’s research. 

Expanding external research impacts

36. Academics should move beyond simply maintaining a CV and publications list and develop and keep updated an ‘impacts file’ which allows them to list occasions of influence in a recordable and auditable way.
Departments and universities can best encourage academics to collation raw data on external impacts by asking staff systematically and regularly about their impact work and assigning some weight to impact track records
in academic promotion procedures.

7. The type of output chosen affects citation rates e.g. on average a book will take longer to be referred to but will be cited for longer.

8. How academics balance their time across the six areas of responsibility will be another important factor in citation rates.

Knowing your strengths

9. In the past academics have had few available tools to track their citation rates. We suggest using a combination of the three best tools which are Harzing’s Publish or Perish, Google Scholar and Google Book Search, and the ISI Web of Knowledge.

       Google Scholar Citations also is a new facility that may have extensive impact and take-up in academia in the next 2-3 years.

10. Having a distinctive author name is very helpful for academics’ work to be easily found amongst a global deluge of information – academics should choose an author name that will allow their citations to be easily retrieved in global-scale publishing markets.

11. Conventional citation-tracking systems like ISI WOK and Scopus have limited coverage in the social sciences and humanities, and an American-based geographical bias, as well as capturing relatively few citations in languages other than English. They are only just planning to cover books and do not cover working papers.

12. Internet-based systems like Harzing’s Publish or Perish, Google and Scirus cover a wider range of academic outputs and now provide more reliable analysis of how research is being cited – they are much more reliable and inclusive in the social sciences and humanities.

Key measures of academic influence

13. Simple indicators for judging citation rates – such as total number of publications, total number of citations, and an age-weighted citation rate do not accurately capture an academics’ citation success.

14. Calculating an academic’s h-score and g-score provides a more robust picture of how much an academic’s work is valued by her peers.

15. Across all disciplines in the social sciences, journal articles account for the majority of citations, reflecting the large numbers of published articles. Books account for 8 to 30 per cent of citations across different disciplines. Books may figure disproportionately amongst those well-cited entries that build h scores and the g index. Book chapters in edited books, however, are often hard to find and are poorly referenced.

16. Network analysis can help shed light on the difference in citation rates between ‘hub’ and ‘authority’ academics at different stages in their careers, which compares the number of inward and outward citations.

Getting better cited

17. Academics who wish to improve the citation rate of their journal articles should ensure that title names are informative and memorable, and that their abstracts contain key ‘bottom line’ or ‘take-away points’. ‘Narrative’ titles that in some way cue findings have many advantages in modern academic communication.

18. Book authors should ensure that their titles and sub-titles are distinctive yet appear in general ‘Google Book’ searches around the given theme.

19. There are a number of schools of thoughts regarding self-citations. In general academics should aim to ensure their own self-citation rate is in line with those of other academics in the same discipline.

20. Co-authored outputs tend to generate more citations due to networking effects between authors in a given research team or lab, especially if the co-authors come from different universities or countries.

Patterns of external research impacts

21. Generating impact within single academic disciplines is a complex process encompassing not only ‘discovery’ but also integration, application, and professional renewal; each of which make significant demands on an academic’s time and contribute to impacts. E.g. some of the academic impacts with the widest reach and greatest significance arise from integration work, not just applications or discovery work, especially in the social sciences

22. Academic work is highly siloed into disciplines while societal problems are multi-dimensional. Bridging scholarship across disciplines, promoting integration at the university level, and engaging in academic and professional service are some key additional ways in which academics’ work can better reach and influence wider society.

other organisations – aggregate, distil and re-package trends in academic research for clients and other actors in the private sector, government, and civil society.

24. Academics giving informal advice to businesses, along with lectures, networking, contract work, student placements, joint publications and consultancy are the most widely undertaken activities likely to generate external impacts.

Is there an impacts gap?

25. Government officials and businesses often complain of an ‘impact gap’ where academic research fails to fulfil its potential to influence wider societal development. (The wider issue of ‘outcome gaps’ is too difficult to track or discuss due to the multi-causal nature of social life and the weak existing evidence base about such issues).

26. If there is an impacts gap it could be attributed to:

·        demand and supply mismatches;

·        insufficient incentives problems;

·        poor mutual understanding and communication;

·        cultural mismatch problems; or

·        weak social networks and social capital.

27. Solutions to effectively combat an impacts gap cannot be homogenous across all academic disciplines and sectors, but rather should be innovative and tailored to the demonstrated problem.

How researchers achieve external impacts

28. While different authors and schools of thoughts within disciplines will take a different view of what factors make a difference to an academic achieving external impacts, we hypothesize that the following seven variable are the most relevant:

·        his or her academic credibility;

·        dispositional and sub-field constraints that rule some academics out of influence;

·        networking skills;

·        personal communication capacity;

·        external reputation;

·        experience of applied and joined-up work; and

·        track record of successful applied and joined-up work.

29. Analysis of our pilot sample of 120 academics shows that academics who are cited more in the academic literature in social sciences are also cited more in non-academic Google references from external actors.

30. Researchers responding to research councils and funding bodies tend to claim impact in a haphazard way. It is possible to see in more detail analysis a more robust link between outputs produced for a particular project and objectively moderated impact assessments.

How organizations achieve external impact

31. While academic departments, labs, and research groups produce a great deal of explicit knowledge, it is their collective ‘tacit knowledge,’ which is the most difficult to communicate to external audiences, and yet that tends to have the most impact.

32. The changing nature of commissioned academic work means that the time lag in achieving external impacts can be radically reduced, yet any external impact of non-commissioned work is likely to lag far beyond its academic impact.

33. It is important for both individual departments/ research labs, schools or faculties, and the University as a whole to systematically collect, access and arrange auditable data on external impacts; keeping in mind that some ‘naïve customers’ like funders, regulators, and other parts of their universities may insist on proof of ‘extended’ impacts

34. Making meaningful comparisons between universities’ and individual departments’ external impacts requires contextual understanding of how departments and universities generally perform in a given country and institutional environment.

35. Seeking to improve external impact should not mean sacrificing academic independence and integrity. Compiling a risk assessment for working with external actors or funders is one way to mitigate the politicization of one’s research.

Expanding external research impacts

36. Academics should move beyond simply maintaining a CV and publications list and develop and keep updated an ‘impacts file’ which allows them to list occasions of influence in a recordable and auditable way.

Departments and universities can best encourage academics to collation raw data on external impacts by asking staff systematically and regularly about their impact work and assigning some weight to impact track records in academic promotion procedures.
37. Universities’ events programmes should be re-oriented toward promoting their own research strengths, as well as external speakers. Events should be integrated multi-media and multi-stage from the outset, and universities should seek to develop ‘zero touch’ technologies to track and better target audience members.
38. Universities should learn from corporate customer relationship management (CRM) systems to better collect, collate, and analyse information gathered from discrete parts of the university and encourage academics to record their impact-related work with external actors. Recommendation 36 above is a key prior stage here.
39. ‘Information wants to be free.’ Publishing some substantive and informative form of an academic research on the open web or storing it in a university’s online depository is essential to ensure that readers beyond academia can gain easy access to research.
40. Improving professional communication, such as through starting multi-author blogs, can help academics ‘cut out the middleman’ and disseminate their research more broadly. By fostering direct and timely communication with wider and more diverse audiences, in a completely open-web format, multi-author academic blogs can greatly expand the detail, accuracy and immediacy of the messages that get ‘translated’ by external audiences.
41. Academics must realise that key interface bodies like think tanks, consultancies and multiple professional networks are not going to go away. Being ‘smart but tough’ about working with intermediaries and networks can broaden access to the potential beneficiaries of research.

Executive Summary | Impact of Social Sciences