Reporting Quality of Diagnostic Accuracy Studies in Laboratory Medicine: Adherence to Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015
2020; 40(3): 245-252
Ann Lab Med 2022; 42(3): 321-330
Published online May 1, 2022 https://doi.org/10.3343/alm.2022.42.3.321
Copyright © Korean Society for Laboratory Medicine.
Department of Parasitology and Institute of Medical Education, Hallym University, Chuncheon, Korea
Correspondence to: Sun Huh, M.D., Ph.D.
Department of Parasitology and Institute of Medical Education, Hallym University, 1 Hallimdaehka-gil, Chuncheon 24252, Korea
Tel: +82-33-248-2652
Fax: +82-33-256-3426
E-mail: shuh@hallym.ac.kr
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background: A bibliometric analysis of the Annals of Laboratory Medicine (ALM) was performed to understand its position in the medical laboratory technology category and to suggest a developmental strategy.
Methods: Journal metrics, including the number of articles by publication type, country of authors, total citations, 2-year impact factor, country of cited authors, journals citing ALM, and Hirsch-index, were obtained from the Journal Citation Report and Web of Science Core Collection. Target data included ALM content in the Web of Science from January 1, 2012, to October 5, 2021. Bibliometric analysis was performed using Biblioshiny.
Results: The impact factor increased from 1.481 in 2013 to 3.464 in 2020. Authors belonging to the USA, China, and Korea cited ALM articles the most. Plos One, Scientific Reports, and Frontiers in Microbiology most frequently cited ALM, besides ALM itself. The Hirsch-index was 34. The co-occurrence network of Keyword Plus indicated four clusters: diagnosis, identification, prevalence, and risk. The conceptual structure map of Keyword Plus based on multiple correspondence analysis showed two clusters: bacterial susceptibility at the bench and clinical courses. The co-citation network showed that ALM was in the cluster of the New England Journal of Medicine, The Lancet, JAMA, and the Annals of Internal Medicine. The collaboration network showed that Korean authors collaborated mainly with authors from the USA, Germany, and Italy.
Conclusions: The journal’s promotion to an international top-tier journal has been successful. “Principles of transparency and best practice in scholarly publishing” and a preprint policy are yet to be added.
Keywords: Bibliometrics, Journal impact factor, Medical laboratory technology, Annals of Laboratory Medicine
The year 2021 signifies the 41st anniversary of the official society of journal publication. The journal’s development in these 41 years has only been possible due to the hard work of the editors and society members. The journal’s present position is excellent among the 29 SCIE journals in the medical laboratory technology category. In this category, three journals, along with ALM, were published by academic societies in the 2020 Journal Citation Report, the
It is time to reflect on ALM’s promotion to a top-tier journal and suggest a strategy to strengthen its leadership based on bibliometric analysis, using journal metrics, document network, conceptual structure, intellectual network, and social structures. Authors, documents, journals, keywords, or authors’ countries can be the subjects of such network analyses [2] to verify the collaboration among researchers, institutes, or countries [3, 4]. Bibliometric analysis also clarifies the evolution of a specific research field [5, 6].
The aim of this study was to clarify the position of ALM in the medical laboratory technology category and to suggest a developmental strategy based on bibliometric analysis. Specifically, the following factors were considered: (1) the change in journal metrics, including the number of articles published, country of authors, country of citing authors, citing journal titles, impact factor, and Hirsch index (h-index), from 2007 to the present; (2) the document network (word cloud and trends topics) and its conceptual structure (co-occurrence network, thematic evolution, and factorial network), intellectual network of cited journals (co-citation network), and country–level social structures (collaboration network) of ALM; and (3) suggestions for developmental strategies based on the above analyses.
A search in Journal Citation Ranking (JCR) and the Web of Science Core Collection® was performed on October 5, 2021. The target journal was ALM. Articles published from 2012 to October 5, 2021, were included. All 1,049 articles were selected and downloaded with all fields, including author, affiliation, title, source, language, document type, keywords, Keywords Plus (Clarivate, London, UK), abstract, and references. Data of 7,399 articles citing ALM were downloaded from JCR. Both data sources were used for the bibliometric analysis.
Journal metrics, including number of articles by publication type, country of authors, total citations, two-year impact factor, country of citing authors, journal titles citing ALM, and h-index were obtained from the Journal Citation Report and Web of Science Core Collection. The h-index is defined as the number of papers with citation number ≥h, with index h reached if the number of papers published over n years (Np) have at least h citations each and the other (Np–h) papers have ≤h citations each [7]. Target data for bibliometric analysis were ALM content in the Web of Science from 2012 to October 5, 2021. Bibliometric analysis was conducted using Biblioshiny, an app version tool of Bibliometrix [8].
Descriptive statistics are used to present and summarize the results.
The number of articles published in ALM by publication type from 2012 to 2022 is presented in Fig. 1. Overall, the total number of original articles published in this period was 635 (60.5%), followed by 322 (30.7%) letters, 50 (4.8%) reviews, 22 (2.1%) editorial materials, and 20 (1.9%) corrections.
Out of the total 1,049 articles published, 825 (78.7%) were written by authors in Korea, followed by authors from China (50, 4.8%), the United States (42, 4.0%), Italy (24, 2.3%), Germany (22, 2.1%), and Japan (21, 2.0%) (Supplemental Data Fig. S1).
The total number of citations was 7,399, which became 7,067 after excluding self-citations. Therefore, the total self-citation frequency was 3.71% (271/7,339). The annual changes in the total citations and number of publications are presented in Supplemental Data Fig. S2.
The 2013 impact factor of the journal was 1.481 [journal impact factor (JIF) 43.6%]. Subsequently, although there were some fluctuations, the impact factor reached 3.464 (JIF 63.7%) (Fig. 2).
The authors of articles that most frequently cited ALM were from the United States (1,340, 18.6%), followed by China (1,115, 16.0%) and Korea (1,096 15.2%). Italy, Germany, France, Japan, England, Turkey, and Spain were also represented among these authors in descending order. Authors from a total of 145 countries/regions cited ALM articles (Supplemental Data Fig. S3).
A total of 2,125 journals cited ALM articles. The journal that most frequently cited ALM was ALM itself (3.7%), followed by
The h-index was calculated to be 34. The 36 most frequently cited articles are listed in Supplemental Data Table S1. The publication type of the most frequently cited article was a review article, titled “Update on procalcitonin measurements,” published in 2014, which had been cited 174 times. Out of the 36 most frequently cited articles, 12 (33.3%) were review articles and 24 (66.7%) were original articles. The authors of 21 of the 36 articles (58.3%) were affiliated to Korean institutions (Supplemental Data Table S1).
In Biblioshiny, the word occurrence measure was set to frequency, and the number of words was set to 50. The most frequent words (Keyword Plus) were diagnosis (68), infection (53), identification (49), gene (39), and mutations (36). The word cloud based on the 50 most frequent words is presented in Fig. 4. Trend topics were analyzed for Keyword Plus. The word minimum frequency, number of words per year, and word label size were each set to 5. Fig. 5 shows the trend topics by year. In 2020, DNA, marker, and β-lactamase were the top trend topics.
For document analyses, Keyword Plus was selected because “the data in Keywords Plus are words or phrases that frequently appear in the titles of an article’s references but do not appear in the title of the article itself, and Keywords Plus enhances the power of cited-reference searching by searching across disciplines for all articles that have cited references in common” [9].
In Biblioshiny, the following conditions were set for the co-occurrence network: field, Keyword Plus; network layout, automatic layout; normalization; association, clustering; algorithm, Louvain; number of nodes, 50; remove isolated node, yes; and minimum edges, 4. The co-occurrence network is presented in Supplemental Data Fig. S4. The first cluster included diagnosis, infection, disease, classification, guidelines, cells, and biomarkers. The second cluster included risk and association.
For the thematic evolution analysis, the following conditions were set: field, Keyword Plus; number of words, 250; minimum cluster frequency (per thousand documents), 5; weight index, inclusion index weighted by word occurrences; minimum weight index, 0.1; number of cutting points, 1; and cutting year, 1 (2017). The map of thematic evolution is presented in Fig. 6. From Keyword Plus of diagnosis, mutation, gene, identification, and system in 2012–2017, I found an evolution to diagnosis, assay, children, expression, classification, risk, identification, COVID-19, performance, gene, and resistance in 2018–2022.
The following conditions were set for the factorial analysis: method, multiple correspondence analysis; field, Keyword Plus; number of terms, 50; number of the cluster, auto. The conceptual structure map is presented in Fig. 7. Bacterial susceptibility at the bench formed one cluster, and clinical courses formed other clusters.
The following conditions were set for the co-citation network: network layout, automatic; clustering algorithm, Louvain; number of nodes, 50; removing isolate nodes, yes; and minimum edges, 5. The co-citation network is presented in Fig. 8. ALM was included in the same cluster as the
The following conditions were set for the collaboration network: field, country; normalization, no; network layout, automatic; clustering algorithm, Louvain; number of nodes, 50; remove isolated nodes, yes; and minimum edge, 2. Fig. 9 presents the collaboration network of the authors’ countries. The leading country was Korea, followed by Germany, the United States, Italy, China, Australia, Japan, and United Arab Emirates. Switzerland and France were included in the same collaboration group. The third collaboration was between India and Saudi Arabia.
The average number of articles published in ALM per year was 103 from 2012 to 2021. Three-quarters of articles were from Korean authors. The number of total citations increased rapidly from 22 in 2012 to 1,697 in 2020. The impact factor also increased from 1.481 in 2013 to 3.464 in 2020. The top three countries of authors who cited ALM were the United States, China, and Korea.
The journal metrics were visualized as figures for clear interpretation. The increase in the total number of citations and impact factor was particularly impressive (Fig. 2, Supplemental Data Fig. S2). Out of four academic society journals in the laboratory medicine technology category, only the
Although authors of ALM are mainly from Korea (78.6%), the countries of the authors most frequently citing ALM were the United States (18.6%) and China (16.0%), followed by Korea in the third place (15.2%) (Supplemental Data Fig. S1). There were citations from 144 countries or regions in total, indicating that the content of ALM was essential to researchers worldwide. Since there is no hurdle to access ALM, it is easy for worldwide researchers to read and cite ALM articles. Furthermore, 2,125 journals listed in the Web of Science Core Collection had cited ALM. This means that most biomedical journals cited ALM and that ALM contributed to the publication of these journals. Mega-journals, including
The word cloud (Fig. 4) provides the content of the journal at a quick glance. The most frequently appearing words “diagnosis,” infection,” and “identification” reflect the scope of ALM, which is the “etiology, diagnosis, and treatment of diseases.” The trending topics (Fig. 5) and thematic evolution map (Fig. 6) provide the yearly changes of the main topics. The low frequency and variety of trend topics in 2019 and 2020 indicate that the topics became more diversified. Owing to the development of diagnostic methods for various diseases, topics will be consistently diverse. The co-occurrence network of Keyword Plus showed four clusters. The “infection and diagnosis” cluster was the largest, and “identification and strains” was the next central cluster. This also reflects the scope of the journal well. The conceptual structure map of Keyword Plus provided one large cluster of clinical courses and another small cluster of bacterial susceptibility at the bench (Fig. 7), which reflect the central concept and the sub-concepts of journal articles.
The co-citation network of the cited journals in ALM was grouped into four clusters (Fig. 8). Co-citation is defined as a linkage between a pair of documents concurrently cited by a third document. Co-cited documents have common topics or subjects used to select the information that can build consilience about ideas and constructs [10]. ALM’s inclusion in the same cluster with the
Some previously published articles also reported bibliometrics or journal metrics for medical journals in Korea. In
The journal has shown outstanding performance considering the number of articles published per year among the 29 SCIE journals in the medical laboratory technology category. Further development is suggested below to promote the journal to a high-grade top-tier international journal based the results of this study.
First, further collaboration of Korean authors with foreign researchers is strongly recommended and should be accelerated through collaborative research or data sharing. The results shown in Fig. 9 demonstrate that the ALM authors’ network with foreign authors remains weak. Therefore, collaborations with researchers of other countries are essential for a clinical trial or diagnostic development. Besides Korean researchers visiting foreign institutes, invitation of foreign researchers to Korea, especially the younger generation, should be more actively initiated for continuous international collaboration. The Official Development Assistance program might be the best way to recruit foreign researchers. These efforts will be reflected as more frequent collaborative work in the journal.
Second, more active recruitment of papers from other countries is needed. Three-quarters of the target papers were by Korean authors (Supplemental Data Fig. S1). The diversity of authors’ countries would reflect the internationality of the journal. Encouraging researchers in Southeast Asia to publish their research work in ALM and a strong editorial support may help increase articles published by these authors.
Third, recent (2018–2021) topics were found to be more diverse than topics from 2012 to 2017 (Fig. 6). Assay, children, expression, risk, COVID-19, performance, and resistance were identified as major topics. ALM has played a strong role in combatting COVID-19 by publishing invaluable papers [2]. Therefore, it is recommended to keep up with the most critical research scope continuously.
Other efforts are also suggested for promoting the journal to a top-tier journal internationally. First, compliance with “principles of transparency and best practice in scholarly publishing” is recommended to be posted as a separate policy on the journal web and print versions, as is the case for
Second, adoption of a preprint policy should be announced to clarify the acceptance of the submission of preprints and preprint references. Preprints refer to pre-published papers uploaded to a public server by the author. Readers of preprint manuscripts can make comments. Subsequently, preprints can usually be submitted to journals. Out of the 383 SCIE journals published by academic societies in Asia, 28 journals accepted preprint submissions and eight allowed authors to cite preprints in the reference list in 2021 [23]. Of the 365 Korean editors and researchers, 230 (63.8%) agreed with accepting preprints in journal publishing [24]. The value of a preprint has already been highlighted not only in the science field but also in the social science field [25]. Accepting a preprint submission is not mandatory; however, the editor or publisher should clearly mention the journal’s preprint acceptance policy.
It is uncertain if “principles of transparency and best practice in scholarly publishing” and the announcement of a preprint policy would help promote the journal’s performance in the near future. However, these policies should be considered to make the brand an international top-tier journal.
The current position of ALM in the journal network is compatible with that of top-tier journals based on the citation analysis. The scope of the journal is also well presented by the document network. The document’s conceptual structure showed two clusters of content: bacterial susceptibility at the bench and clinical courses. According to the intellectual structure of cited journals, ALM was included in the cluster with highly influential journals such as the
None.
All work was performed by the corresponding author.
No potential conflicts of interest relevant to this article are reported.
This work was supported by the Hallym University Research Fund (HRF-202002190001).