Come valutare la ricerca nelle Aziende Ospedaliero-Universitarie? Metodi e proposte operative

Journal title MECOSAN
Author/s Sara Barsanti, Silvia Giovanelli, Sabina Nuti
Publishing Year 2014 Issue 2014/90
Language Italian Pages 24 P. 9-32 File size 424 KB
DOI 10.3280/MESA2014-090002
DOI is like a bar code for intellectual property: to have more infomation click here

Below, you can see the article first page

If you want to buy this article in PDF format, you can do it, following the instructions to buy download credits

Article preview

FrancoAngeli is member of Publishers International Linking Association, Inc (PILA), a not-for-profit association which run the CrossRef service enabling links to and from online scholarly content.

At international level, the measurement of the Teaching Hospitals (THs) performance is mainly focused on clinical quality and on major costs with regards to teaching and non teaching settings; even if research activities represent one of the three main issues that define the triple mission of these institutions, they are rarely evaluated. Using Tuscan THs as case studies, the authors propose a set of bibliometrics indicators based on the Impact Factor (IF) to systematically evaluate research outputs of the THs. Despite, the international debate on the IF is still ongoing, this is the most common indicator to measure research activities, considering the availability, the objectivity and the feasibility of its calculation.

Keywords: Scientific productivity, research, evaluation, Teaching Hospital, Impact Factor, bibliometric indicators

  1. Catalfo P., Cavalieri M., Guccio C. (2011). Finanziamento nei teaching hospitals: tra risorse immateriali e sistema di misurazione della performance. Mecosan, 77: 51-74.
  2. Abernethy M.A., Stoelwinder J.U. (1995). The role of professional control in the management of complex organizations. Accounting, organizations and society, 20 (1): 1-17. DOI: 10.1016/0361-3682(94)E0017-O
  3. Abernethy M.A., Stoelwinder J.U. (1990). The relationship between organization structure and management control in hospitals: an elaboration and test of Mintzberg’s professional bureaucracy model. Accounting, Auditing & Accountability Journal, 3 (3): 18-33. DOI: 10.1108/09513579010142616
  4. Abernethy M.A., Vagnoni E. (2004). Power, organization design and managerial behavior. Accounting, organizations and society, 29: 207-225. DOI: 10.1016/S0361-3682(03)00049-7
  5. Aidemark L.G. (2001). The meaning of balanced scorecards in the healthcare organization. Financial accountability & management, 17 (1): 23-40. DOI: 10.1111/1468-0408.00119
  6. Baccini A. (2010). Valutare la ricerca scientifica. Uso e abuso degli indicatori bibliometrici. Bologna: il Mulino/Ricerca.
  7. Baraldi S. (2005). Il balanced scorecard nelle aziende sanitarie. Milano: McGraw-Hill Companies. Barresi G., Cardamone E., Cavallaro G., Mauro
  8. M., Marisca C., Trotta A. (2010). La misurazione della performance delle AOU. Il caso degli ospedali di insegnamento in Calabria e in Sicilia. Working Paper presentato al XXXIII Convegno Annuale AIDEA Pubblico e non profit per un mercato responsabile e solidale, Università Bocconi, 21-22 ottobre 2010.
  9. Barsanti S., Sanna F., Murante A.M., Calacrese C., Camerini V. (2008). La valutazione della capacità di perseguimento degli obiettivi regionali. In: Nuti S. (a cura di). La valutazione della performance nella sanita. Bologna: il Mulino.
  10. Borgonovi E. (2010). Postfazione. In: Carbone C., Lega F., Prenestini A. (a cura di). Governance e organizzazione delle aziende-ospedaliere universitarie. Milano: Egea.
  11. Carbone C., Lega F., Prenestini A. (a cura di) (2010). Governance e organizzazione delle aziendeospedaliere universitarie. Milano: Egea.
  12. Carbone C., Lega F., Salvatore D., Tozzi V. (2007). Ospedali e università: illusioni, delusioni e realtà di un rapporto difficile. In: Anessi Pessina E., Cantù E. (a cura di). L’aziendalizzazione della sanita in Italia. Rapporto Oasi. Milano: Egea.
  13. Cassella M., Bozzarelli O. (2011). Nuovi scenari per la valutazione della ricerca tra indicatori bibliometrici citazionali e metriche alternative nel contesto digitale. Biblioteche Oggi, 29 (2): 66-78.
  14. De Negri A., Montaguti U., Favero L., Callea G. (2005). Benchmarking e “bilanciamento” qualitàcosti. Un’esperienza nazionale relativa alle aziende ospedaliere universitarie. In: Barretta A., Vagnoni
  15. E. (a cura di). Il benchmarking in ambito sanitario: esperienze a confronto. Padova: Cedam. Decreto legislativo 16 ottobre 2003, n. 288. Decreto legislativo 30 dicembre 1992, n. 502.
  16. Derrick G.E., Haynes A., Champman S., Hall W.D. (2011). The association between four citation metrics and peer rankings of research influence of Australian researchers in six fields of public health. Plos one, 6 (4): e18521. DOI: 10.1371/journal.pone.0018521
  17. DiMaggio P.J., Powell W.W. (1983). The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. American sociological review, 48: 147-160. DOI: 10.1016/S0742-3322(00)17011-1
  18. Emmanuel J., Favaloro E.J. (1998). Medical research in New South Wales 1993-1996 assessed by medline publication capture. Medical Journal of Australia, 169 (11): 617-622.
  19. Falagas M.E., Papastamataki P.A., Bliziotis I.A. (2006). A bibliometric analysis of research productivity in parasitology by different world regions during a 9-year period (1995-2003). BCM infectious diseases, 6: 56. DOI: 10.1186/1471-2334-6-56
  20. Fitzpatrick J.M., Shamim Khan M., Dasgupta P. (2012). Indicators for research performance evaluation: an overview. BJU International, 109: 321-326. DOI: 10.1111/j.1464-410X.2011.10856.10881.x
  21. Froghi S., Ahmed K., Finch A., Fitzpatrick J.M., Khan M.S., Shamim Khan M., Dasgupta P. (2012). Indicators for research performance evaluation: an overview. BJU International, 109 (3): 321-326. DOI: 10.1111/j.1464-410x.2011.10856,10881.x
  22. Glouberman S., Mintzberg H. (2001). Managing the care of health and the cure of disease. Part 2. Integration. Health Care Management Review, 26: 70-87.
  23. Hirsch J.E. (2005). An index to quantify an individual’s scientific research output. Testo disponibile al sito:
  24. Ho Y.S., Chiu C.H., Tseng T.M., Chiu W.T. (2003). Assessing stem cell research productivity. Scientometrics, 57 (3): 369-376. DOI: 10.1023/A:1025004701748
  25. Kaltenborn K.F., Kuhn K. (2004). The journal impact factor as a parameter for the evaluation of researchers and research. Revista Espanola de Enfermedades Digestivas (Madrid), 96 (7): 460-476. DOI: 10.1007/s00063-003-1240-6
  26. Kaplan R. S. and Norton D.P. (1992). The Balanced Scorecard: Measures that Drive Performance. Harvard Business Review, ( January-February): 71-79.
  27. Kurunmäki L. (2004). A hybrid profession: the acquisition of management accounting expertise by medical professionals. Accounting, organization and society, 29: 327-347. DOI: 10.1016/S0361-3682(02)00069-7
  28. Lega F. (2008). Management dell’azienda brainintensive. Milano: Egea. Michalopoulos A., Bliziotis I., Rizos M., Falagas
  29. M.E. (2005). Worldwide research productivity in critical care medicine. Critical Care, 9: r258-r265.
  30. Mintzberg H. (1979). The structuring of organizations. Englewood Cliffs, NJ: Prentice-Hall. Naranjog D., Hartmann F.G.H. (2004). Leadership versus management accounting systems: an exploratory study in hospitals. Management Control & Accounting, 6: 26-31.
  31. Numerato D., Salvatore D., Fattore G. (2012). The impact of management on medical professionalism: a review. Sociology of health and illness, 34 (4): 626-644. DOI: 10.1111/j.1467-9566.2011.01393.x
  32. Nuti S., Bonini A., Vainieri M. (2010). Disinvestment for reallocation: a process to identify priorities in healthcare. Health policy, 95(2-3): 137-143.
  33. Nuti S. (a cura di) (2008). La valutazione della performance in sanita. Bologna: il Mulino.
  34. Orlikowski W.J. (1991). Integrated information environment or matrix of control? The contradictory implications of information technology. Accounting management and information, 1: 9-42. DOI: 10.1016/0959-8022(91)90011-3
  35. Ouchi W.G. (1977). The relationship between organizational structure and organizational control. Administrative Science Quarterly, 22 (1): 95-113.
  36. Ouchi W.G., Price R.L. (1978). Hierarchies, clans and theory: a new perspective on organization development. Organizational Dynamics, 7: 62-70. DOI: 10.1016/0090-2616(78)90036-0
  37. Ouchi W.G. (1979). A conceptual framework for the design of organizational control mechanism. Management Science, 25 (9): 833-848.
  38. Ouchi W.G. (1978). The transmission of control through organizational hierarchy. Academy of Management Journal, 21 (2): 173-192. DOI: 10.2307/255753
  39. Patel V.M., Ashrafian H., Ahmed K., Arora S., Jiwan S., Nicholson J.K., Darzi A., Athanasiou T. (2011).
  40. How has healthcare research performance been assessed? A systematic review. Journal of the Royal Society of Medicine, 104: 251-261. DOI 10.1258/jrsm.2011.110005.
  41. Peters J., Marsh R. (2009). Rate my research dot com: measuring what we value, and valuing what we measure. Management Decision, 47 (9): 1452-1457. DOI: 10.1108/00251740910995657
  42. Pons J., Sais C., Illa C., Méndez R., Suñen E., Casas M., Camí J. (2010). Is there an association between the quality of hospitals’ research and their quality of care? Journal of health services research & policy, 5 (4): 204-209. DOI: 10.1258/jhsrp.2010.009125.Epub2010May13
  43. Powell W., DiMaggio P.J. (1991). The new institutionalism in organizational analysis. Chicago: University of Chicago Press.
  44. Regione Toscana (2005). Testo coordinato della legge regionale 24 febbraio 2005, n. 40. Disciplina del servizio sanitario regionale.
  45. Regione Toscana (2011). DGRT 398/2011, Individuazione e assegnazione obiettivi alle Aziende Sanitarie Toscane per farmaci e dispositivi medici per l’anno 2011.
  46. Regione Toscana (2013). DGRT 201/2013, Individuazione e assegnazione obiettivi alle Aziende Sanitarie Toscane per farmaci e dispositivi medici per l’anno 2013.
  47. Rehn C., Kronman U. (2008). Bibliometric indicators – definition and usage at Karolinska Institutet. Testo disponibile al sito iles/bibliometric_indicators_definitions_1.0.pdf (ultimo accesso febbraio 2014).
  48. Rockness H., Shields M. (1988). An empirical analysis of the expenditure budget in research and development. Contemporary accounting research, 4 (2): 568-581 DOI: 10.1111/j.1911-3846.1988.tb00685.x
  49. Rodríguez-Ruiz O. (2009). The citation indexes and the quantification of knowledge. Journal of educational administration, 47 (2): 250-266. DOI: 10.1108/09578230910941075
  50. Rosmarakis E.S., Vergidis P.I., Soteriades E.S., Paraschakis K., Papastamataki P.A., Falagas M.E. (2005). Estimates of global production in cardiovascular diseases research. International Journal of Cardiology, 100: 4439. DOI: 10.1016/j.ijcard.2004.11.005
  51. Ruef M., & Scott W. R. (1998). A multidimensional model of organizational legitimacy: Hospital survival in changing institutional environments. Administrative Science Quarterly, 43(4): 877-904.
  52. Sahel J.A. (2011). Quality versus quantity: assessing individual research performance. Science translation medicine, 3 (84): 84cm13. DOI: 10.1126/scitranslmed.3002249
  53. Santori G., Valente R., Ghirelli R., Valente U. (2008). Evaluation of research products released during a national project funded by the Italian ministry of health. Transplantation Proceedings, 40: 1820-1822. DOI: 10.1016/j.transproceed
  54. 2008.05.017. Schapper C.C., Dwyer T., Tregear W.G., Aitken M., Clay M.A. (2012). Research performance evaluation: the experience of an independent medical research institute. Australian Health Review, 36: 218-223. DOI: 10.1071/AH11057
  55. Schreyögg J., Von Reitzenstein C. (2008). Strategic groups and performance differences among academic medical centers. Health care management review, 33 (3): 225.
  56. Smith T., Whitchurch C. (2002). The future of the tripartite mission: reexamining the relationship linking universities, medical schools and health systems. Higher education management and policy, 14 (2): 39-52. DOI: 10.1787/hemp-v14-art12-en
  57. Soteriades E.S., Falagas E.M. (2005). Comparison of amount of biomedical research originating from the European union and the United States. BMJ, 331: 192. DOI: 10.1136/bmj.331.7510.192
  58. Tediosi F., Compagni A. (2010). Analisi del sistema di finanziamento della ricerca sanitaria in Italia. Milano: FrancoAngeli.
  59. Tess B.E., Furuie S.S., Figuereido Castro R.C., Cavarette Barreto M., Cuce Nobre M.R. (2009). Assessing the scientific research productivity of a brazilian healthcare institution: a case study at the heart institute of Sao Paolo, Brazil. Clinics, 64 (6): 571-576. DOI: 10.1590/S1807-59322009000600013
  60. Topping S., Malvey D. (2002). Management of academic health centers: the past, present, and future. Advances in Health Care Management, 3: 267-297. DOI: 10.1016/S1474-8231(02)03012-4
  61. Trotta A., Cardamone E., Cavallaro G., Mauro M. (2012). Applying the balanced scorecard approach in teaching hospitals: a literature review and conceptual framework. International Journal of Health Planning and Management, 28 (2): 181-201. DOI: 10.1002/hpm.2132
  62. Vergidis P.I., Karavasiou A.I., Paraschakis K., Bliziotis I.A., Falagas M.E. (2005). A bibliometric analysis of global trends of research productivity in microbiology. European Journal of Clinical Microbiology & Infectious Diseases, 24: 3426.

  • Do university hospitals perform better than general hospitals? A comparative analysis among Italian regions Sabina Nuti, Tommaso Grillo Ruggieri, Silvia Podetti, in BMJ Open /2016 pp.e011426
    DOI: 10.1136/bmjopen-2016-011426
  • The Italian interregional performance evaluation system Federico Vola, Vera Benedetto, Milena Vainieri, Sabina Nuti, in Research in Health Services & Regions 10/2022
    DOI: 10.1007/s43999-022-00010-6
  • Diversità di genere e performance organizzativa: un'analisi empirica nel settore sanitario Valeria Naciti, Guido Noto, Carlo Vermiglio, in MECOSAN 120/2022 pp.45
    DOI: 10.3280/MESA2021-120004

Sara Barsanti, Silvia Giovanelli, Sabina Nuti, Come valutare la ricerca nelle Aziende Ospedaliero-Universitarie? Metodi e proposte operative in "MECOSAN" 90/2014, pp 9-32, DOI: 10.3280/MESA2014-090002