Performance Evaluation of a Respiratory Virus Multiplex Real-Time PCR Panel Including SARS-CoV-2 and Its Clinical Utility in the Post-Pandemic Period
Article information
Abstract
Since severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) initially appeared, the diagnostic approach to respiratory viruses has evolved significantly. With the advent of the post-pandemic era, the prevalence of non–SARS-CoV-2 respiratory viruses has increased, underscoring the need for comprehensive diagnostic tools. We tested the performance of the PowerCheck Respiratory Virus Panel (Kogene), a multiplex real-time polymerase chain reaction (PCR) assay that includes SARS-CoV-2, on 267 residual nucleic acid samples collected during and after the pandemic. The results were compared with those from the single-target SARS-CoV-2 real-time PCR assays. The detection rates of non-SARS-CoV-2 in isolation as well as their detection rates in patients infected with SARS-CoV-2 respiratory viruses were also analyzed. The PowerCheck Respiratory Virus Panel demonstrated an overall agreement of 95.88%, with a non-negative agreement of 97.18% and a negative agreement of 98.89% with the reference SARS-CoV-2 real-time PCR. Although the difference was not statistically significant, the codetection rate of SARS-CoV-2 with other respiratory viruses was greater in the post-pandemic period. Detection of non–SARS-CoV-2 respiratory viruses in SARS-CoV-2–negative samples was also more frequent following the pandemic. The PowerCheck Respiratory Virus Panel reliably detected SARS-CoV-2. In the post-pandemic era, where multiple respiratory viruses are circulating, multiplex PCR assays targeting a diverse spectrum of pathogens may exhibit greater clinical utility than SARS-CoV-2–specific PCR alone.
Introduction
Since the breakout of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in late 2019, the diagnostic paradigm for respiratory viruses has undergone substantial alterations. During the coronavirus disease 2019 (COVID-19) pandemic, SARS-CoV-2 detection became the primary diagnostic priority. Hence, real-time polymerase chain reaction (PCR) assays targeting exclusively SARS-CoV-2 were widely adopted, and in South Korea, the use of SARS-CoV-2 real-time PCR was also recommended by the national guidelines for the laboratory diagnosis of COVID-19 [1]. Several studies have discovered a marked decline in the positivity rate of tests for detecting respiratory viruses during the pandemic compared with the pre-pandemic period [2-5], suggesting that most patients with respiratory symptoms were infected with SARS-CoV-2. This also reflects the influence of many non-pharmaceutical interventions (NPIs) during the pandemic, which led to a decrease in the incidence of other respiratory virus infections.
In the period following the COVID-19 pandemic, the progressive relaxation of NPIs resulted in alterations in the epidemiological patterns of respiratory viruses [2,6]. The resurgence of multiple respiratory viruses, including influenza, respiratory syncytial virus (RSV), and human metapneumovirus (HMPV), in addition to SARS-CoV-2, is making clinical diagnosis and patient management more complex. In this context, multiplex PCR assays that can simultaneously detect SARS-CoV-2 and other respiratory viruses are becoming more popular as effective tools for comprehensively diagnosing patients presenting with respiratory symptoms during the post-pandemic era [7].
This study assessed the effectiveness of SARS-CoV-2 detection using the PowerCheck Respiratory Virus Panel (Kogene, Seoul, Korea), a recently approved multiplex PCR assay in South Korea that includes SARS-CoV-2 as one of its targets. Results from a commercially available SARS-CoV-2 real-time PCR assay, obtained from 281 residual nucleic acid samples extracted from clinical specimens collected during the pandemic and post-pandemic periods, were compared with the results obtained from testing the same samples using the newly introduced PowerCheck Respiratory Virus Panel. Furthermore, we analyzed the rates of co-detection of SARS-CoV-2 with other respiratory viruses, as well as the detection rates of non-SARS-CoV-2 respiratory viruses during the two periods, to assess the clinical usefulness of multiplex real-time PCR in the post-pandemic era.
Methods
Specimens
The PowerCheck Respiratory Virus Panel was used to evaluate 281 samples of residual nucleic acids. These comprised 231 samples (181 positive and 50 negative for SARS-CoV-2) originally tested utilizing the SARS-CoV-2 real-time PCR assay between February 2020 and December 2023 and 50 samples examined in 2024 using the Real-Q RV Detection Kit (Biosewoom, Seoul, Korea), a respiratory virus multiplex real-time PCR assay that did not target SARS-CoV-2. The annual distribution of the included samples is summarized in Table 1.
Annual distribution of the residual nucleic acid samples included in the study
For the 231 residual nucleic acid samples obtained between February 2020 and December 2023 with pre-existing SARS-CoV-2 real-time PCR results (tested using the Allplex SARS-CoV-2 Assay [Seegene, Seoul, Korea] until February 2021 and the Real-Q Direct SARS-CoV-2 Detection Kit [Biosewoom, Seoul, Korea] thereafter), any differences in SARS-CoV-2 detection between the PowerCheck Respiratory Virus Panel and the original real-time PCR results (e.g., cases where the original result was positive but the multiplex PCR result was indeterminate or negative) were further investigated. In such cases, the same nucleic acid samples were subjected to repeat SARS-CoV-2 real-time PCR testing using a different platform, the Real-Q Direct SARS-CoV-2 Detection Kit.
Both the PowerCheck Respiratory Virus Panel and the Real-Q Direct SARS-CoV-2 Detection Kit were utilized to conduct PCR testing for the 50 remaining nucleic acid samples from 2024 that lacked pre-existing SARS-CoV-2 real-time PCR results. Fourteen of these 50 samples had insufficient nucleic acid volume to be tested with the Real-Q Direct SARS-CoV-2 Detection Kit.
Respiratory virus multiplex real-time polymerase chain reaction using the PowerCheck Respiratory Virus Panel
The PowerCheck Respiratory Virus Panel includes a predispensed strip containing a PCR mixture, a probe and primer mixture, and an enzyme mixture. The respiratory virus multiplex real-time PCR assay was performed employing the PowerCheck Respiratory Virus Panel in accordance with the manufacturer’s instructions. Briefly, 5 μL of extracted nucleic acid was added to each strip tube and transferred to the PowerAmp96 Dx Real-Time PCR System (Kogene, Seoul, Korea). The amplification conditions were as follows: one cycle at 50°C for 15 minutes, one cycle at 95°C for 5 minutes, followed by five cycles of 95°C for 10 seconds and 60°C for 40 seconds, and finally 40 cycles of 95°C for 10 seconds and 60°C for 40 seconds. Each PCR cycle included a positive control provided by the manufacturer and a negative control utilizing sterile, DNase- and RNase-free water.
The PowerCheck Respiratory Virus Panel consists of four separate tubes, each targeting a different category of respiratory viruses. Panel 1 targets influenza A/B viruses and SARS-CoV-2; Panel 2 targets enterovirus/rhinovirus, human coronavirus (HCoV)-229E, HCoV-NL63, and HCoV-OC43; Panel 3 targets human parainfluenza viruses 1, 2, 3, and 4; and Panel 4 targets adenovirus, HMPV, RSV, and bocavirus.
Specifically, Panel 1 comprises tests for two unique genes of SARS-CoV-2: N and ORF1ab. The results are interpreted using a cycle threshold (Ct) cutoff value of 34 for each gene. A positive result is when both target genes exhibit Ct values < 34. If only one gene has a Ct value of < 34, the result is classified as indeterminate. Both positive and indeterminate results were classified as nonnegative.
Severe acute respiratory syndrome coronavirus 2 real-time polymerase chain reaction using the Real-Q Direct SARS-CoV-2 Detection Kit
The Real-Q Direct SARS-CoV-2 Detection Kit also contains a predispensed strip comprising a PCR mixture, a probe and primer mixture, and an enzyme mixture. The real-time PCR assay was performed using the Real-Q Direct SARS-CoV-2 Detection Kit, as per the manufacturer’s instructions. Briefly, each strip tube received 5 μL of extracted nucleic acid before being placed in the CFX96 Real-Time PCR Detection System (Bio-Rad, Richmond, CA, USA). The steps of amplification were as follows: one cycle at 50°C for 10 minutes, one cycle at 95°C for 3 minutes, followed by three cycles of 95°C for 1 second and 62°C for 20 seconds, and lastly 40 cycles of 95°C for 1 second and 62°C for 30 seconds. The manufacturer supplied a positive control for each PCR cycle, and sterile water free of DNase and RNase was used as a negative control.
The Real-Q Direct SARS-CoV-2 Detection Kit targets two specific genes in the SARS-CoV-2 genome: RdRP and E. For each gene, a Ct cutoff value of 38 was used for result interpretation. The result was considered positive when both target genes exhibited Ct values < 38. If either of the genes exhibited a Ct value of < 38, the result was classified as indeterminate. Both positive and indeterminate results were considered as nonnegative.
Data analysis and statistics
The experimental method consisted of analyzing the SARS-CoV-2 results obtained using the PowerCheck Respiratory Virus Panel, while the control method consisted of pre-existing SARS-CoV-2 results from either the previously performed real-time PCR or the newly performed test using the Real-Q Direct SARS-CoV-2 Detection Kit. Implementing the interpretation criteria based on the Ct values of each kit, overall agreement was defined as total concordance between the experimental and control method results in terms of positive, indeterminate, and negative classifications. A non-negative agreement was characterized as cases where the control method result was interpreted as non-negative (positive or indeterminate) and the experimental method result was also non-negative. Negative agreement was defined as cases where both the control and experimental method results were negative.
The COVID-19 pandemic period was defined as the period between 2020 to April 18, 2022, when social distancing regulations were completely repealed in South Korea. The period that followed the pandemic was referred to as the post-pandemic period. To examine detection rates—including the rate of co-detection of SARS-CoV-2 with other respiratory viruses and the rate of detection of other respiratory viruses without SARS-CoV-2—Pearson’s chi-squared test or Fisher’s exact test was employed as appropriate. A p-value of < 0.05 was considered statistically significant. All statistical analyses were performed using the R software environment (version 4.3.1, R Foundation for Statistical Computing, Vienna, Austria).
Results
Performance evaluation of severe acute respiratory syndrome coronavirus 2 detection using the PowerCheck Respiratory Virus Panel
The PowerCheck Respiratory Virus Panel’s efficacy in detecting SARS-CoV-2 was evaluated utilizing 231 residual nucleic acid samples with pre-existing SARS-CoV-2 real-time PCR results. Table 2 presents the comparative results. The overall agreement was 93.51% (216/231; 95% confidence interval [CI]: 89.30–96.19). The PowerCheck Respiratory Virus Panel revealed negative results for every sample that has previously been reported to have negative results using the SARS-CoV-2 real-time PCR. The results of PowerCheck Respiratory Virus Panel and the pre-existing SARS-CoV-2 real-time PCR results demonstrated a 93.92% non-negative agreement (95% CI: 89.39–96.93) and a 100% negative agreement (95% CI: 92.89–100) when the results were categorized as negative or non-negative. With a Cohen’s kappa coefficient of 0.87 (95% CI: 0.80–0.94), this indicates an almost perfect agreement.
Comparison of the results of SARS-CoV-2 detection by the PowerCheck Respiratory Virus Panel (Kogene) in 231 residual nucleic acid samples with the pre-existing SARS-CoV-2 real-time polymerase chain reaction results
Among the 181 residual nucleic acid samples that were positive in the pre-existing SARS-CoV-2 real-time PCR results, 16 samples exhibited a Ct value of ≥ 30 for either the ORF1ab or RdRP/S gene. The disagreement rate in these 16 high-Ct samples was 75% (95% CI: 47.41–91.67), which was significantly greater than that of 1.82% (95% CI: 0.47–5.64) observed in samples with Ct values of < 30 that tested positive (Fig. 1).
Agreement of SARS-CoV-2 detection based on the Ct values in SARS-CoV-2 positive samples. Percentages indicate the proportion of agreement and disagreement cases within each group of positive samples categorized by Ct value. SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; Ct, cycle threshold.
Further analysis was performed using the Real-Q Direct SARS-CoV-2 Detection Kit for the 15 samples that exhibited disagreement with the pre-existing SARS-CoV-2 real-time PCR results (4 positive → indeterminate, 11 positive → negative). All four samples that demonstrated indeterminate results on the PowerCheck Respiratory Virus Panel yielded positive results. Six of the 11 samples that tested negative using the PowerCheck Respiratory Virus Panel remained negative, four were inconclusive, and one tested positive (Table 3).
Additional analysis of 15 samples that yielded inconsistent results when compared to the pre-existing SARS-CoV-2 real-time polymerase chain reaction results
Among the remaining 50 nucleic acid samples, which were not previously tested using SARS-CoV-2 real-time PCR, 14 samples could not be tested due to inadequate nucleic acid volume. Excluding these samples, the overall agreement between the PowerCheck Respiratory Virus Panel and the Real-Q Direct SARS-CoV-2 Detection Kit in the remaining 36 samples was 94.44% (34/36; 95% CI: 79.99–99.03; Table 4).
Results of SARS-CoV-2 detection using the PowerCheck Respiratory Virus Panel (Kogene) in 36 residual nucleic acid samples, which were not previously tested using SARS-CoV-2 real-time polymerase chain reaction assay
The two samples that showed disagreement both yielded indeterminate results when tested using the PowerCheck Respiratory Virus Panel. One of these samples with a Ct value of ≥ 30 tested positive for the RdRP gene using the Real-Q Direct SARS-CoV-2 Detection Kit, whereas the other sample yielded a negative result.
For all 267 samples tested, the overall agreement between SARS-CoV-2 detection by the PowerCheck Respiratory Virus Panel and SARS-CoV-2 real-time PCR was 95.88% (256/267; 95% CI: 92.54–97.82; Table 5). When the results were classified as negative or non-negative, the nonnegative agreement rate was 97.18% (95% CI: 93.53–99.08), and the negative agreement rate was 98.89% (95% CI: 93.96–99.97). With a Cohen’s kappa coefficient of 0.95 (95% CI: 0.91–0.99), this indicates an almost perfect agreement.
Overall efficacy of SARS-CoV-2 detection of the PowerCheck Respiratory Virus Panel (Kogene) in all 267 residual nucleic acid samples
Eleven samples demonstrated disagreement. Five samples that tested positive on the SARS-CoV-2 real-time PCR assay yielded indeterminate results when tested utilizing the PowerCheck Respiratory Virus Panel. Moreover, there were four cases where the result was indeterminate on the SARS-CoV-2 real-time PCR assay but negative by the PowerCheck Respiratory Virus Panel (Table 3). Ct values of > 30 were found in all six samples that tested positive on the SARS-CoV-2 real-time PCR assay but produced indeterminate or negative results when tested using the PowerCheck Respiratory Virus Panel.
Of the 94 samples that tested negative on the PowerCheck Respiratory Virus Panel, seven demonstrated amplification signal values that were below the positive Ct cutoff. The disagreement rate among these seven samples with weak amplification was 57.14% (95% CI: 20.24–88.19), which was significantly greater than the disagreement rate of 1.15% (95% CI: 0.60–7.13) observed in samples without weak amplification (Fig. 2).
Agreement of SARS-CoV-2 detection based on the existence of weak amplification in SARS-CoV-2 negative samples. Percentages indicate the proportion of agreement and disagreement cases within each group of negative samples categorized by the presence or absence of weak amplification. SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
Co-detection rate of other respiratory viruses and detection rate of non-severe acute respiratory syndrome coronavirus 2 respiratory viruses during the pandemic and post-pandemic periods
Among the 281 residual nucleic acid samples tested with the PowerCheck Respiratory Virus Panel, co-detection of SARS-CoV-2 with one or more additional respiratory viruses was observed in six cases (2.14%; 95% CI: 0.87–4.82). The codetection rate among the 173 samples that tested nonnegative for SARS-CoV-2 was approximately 3.47% (95% CI: 1.42–7.74; Fig. 3).
Distribution of respiratory virus detection during the pandemic and post-pandemic periods. Percentages indicate the proportion of codetection cases among all samples tested during each period. SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
Two codetection cases were reported during the pandemic period, accounting for 2.02% (2/99; 95% CI: 0.35–7.81) of the SARS-CoV-2 nonnegative samples. In the post-pandemic period, four cases were identified, corresponding to 5.41% (4/74; 95% CI: 1.75–13.98) of the SARS-CoV-2 nonnegative samples; however, this difference was not statistically significant (p = 0.376). Detection of respiratory viruses other than SARS-CoV-2 was observed in four cases during the pandemic period, accounting for 7.02% (4/57; 95% CI: 2.27–17.83) of the SARS-CoV-2-negative samples. Eleven such cases were identified in the post-pandemic era, corresponding to 21.57% (11/51; 95% CI: 11.75–35.7); however, this difference was also not statistically significant (p = 0.096).
Fig. 4 illustrates the annual distribution of all non-SARS-CoV-2 respiratory viruses identified during the course of this study.
Annual distribution of the detection of non–SARS-CoV-2 respiratory virus. SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; RSV, respiratory syncytial virus; Inf A, influenza A virus; Inf B, influenza B virus; HMPV, human metapneumovirus; HCoV-OC43, human coronavirus OC43; HCoV-NL63, human coronavirus NL63; HCoV-229E, human coronavirus 229E.
Discussion
When comparing the results of PowerCheck Respiratory Virus Panel for the 231 residual nucleic acid samples with the pre-existing results using the SARS-CoV-2 real-time PCR, the SARS-CoV-2 detection performance of the PowerCheck Respiratory Virus Panel revealed an overall agreement of less than 95%. Most of the disagreement cases involved samples that exhibited high-Ct values (≥ 30) in the pre-existing SARS-CoV-2 real-time PCR results. Additionally, further examination of the discordant samples revealed that many exhibited negative results upon retesting. These data reveal that most disagreements may have resulted from nucleic acid degradation due to the prolonged storage of low viral load samples.
After accounting for the additional SARS-CoV-2 real-time PCR testing on the discordant samples, the comparison of SARS-CoV-2 detection between the two PCR assays across all 267 nucleic acid samples showed an increased overall agreement rate of 95.88% with the PowerCheck Respiratory Virus Panel. Notably, the negative agreement rate approached 99%, indicating an excellent performance.
Among the samples that yielded non-negative results on SARS-CoV-2 real-time PCR but tested negative by the PowerCheck Respiratory Virus panel, most displayed weak amplification signal strengths that did not meet the Ct cutoff value. This outcome suggests that the PowerCheck Respiratory Virus Panel rarely misses true non-negative cases and supports its potential utility as a reliable alternative for SARS-CoV-2 detection in clinical laboratories.
Since the initial COVID-19 related symptoms, such as fever, cough, and fatigue, closely resemble those of influenza and the common cold, it can be challenging to differentiate it from other respiratory viruses [8,9]. Therefore, using multiplex PCR assays that target a variety of respiratory viruses during the pandemic may assist in avoiding potential confusion in cases of co-infection [9]. However, prior studies examining consecutive respiratory samples collected during the pandemic period revealed a relatively low co-infection rate of less than 4% in approximately 7% of SARS-CoV-2–positive samples [10,11]. In South Korea, research conducted during the pandemic revealed even lower co-infection rates of approximately 1%–3% among SARS-CoV-2–positive samples, depending on the institution [12,13]. Consistent with previous findings, the current study also discovered a low codetection rate of 2.02% among the SARS-CoV-2 nonnegative samples collected during the pandemic period.
The clinical impact of co-infection with other respiratory viruses in COVID-19 patients remains unclear. Morales-Jadán et al. [14] reported that coinfection with influenza A virus was linked to more severe clinical symptoms, while Tran et al. [15] revealed that coinfection with adenovirus increased the 30-day all-cause mortality. In contrast, Burrel et al. [10] reported that co-infection with other respiratory viruses was significantly associated with a lower rate of hospitalization in the intensive care unit. Although previous studies have shown conflicting results regarding the clinical impact of coinfections, the existence of such influence—regardless of direction—highlights the importance of using respiratory virus multiplex real-time PCR to detect even low rates of coinfection [16].
Although not statistically significant, our study found a greater codetection rate during the post-pandemic period compared with the pandemic period. Additionally, the post-pandemic period exhibited a higher frequency of respiratory virus identification in SARS-CoV-2-negative samples. These findings indicate that employing respiratory virus multiplex real-time PCR may be more beneficial than testing carried out using SARS-CoV-2–specific PCR alone in the post-pandemic era. Moreover, as the seasonal and regional patterns of respiratory viruses have been demonstrated to alter in the post-pandemic period [6,9,17], multiplex PCR testing can be a valuable tool for detecting these alterations and comprehending the evolving distribution of respiratory viruses.
There are various limitations to this study. First, due to limitations in sample availability and laboratory capacity, it was not feasible to perform additional SARS-CoV-2 real-time PCR testing for all stored residual nucleic acid samples. Second, the samples were selected based on availability rather than predefined clinical inclusion criteria. Due to resource constraints, it was not feasible to test all residual samples, and efforts were made to distribute the samples evenly across the study years; however, the heterogeneity in sample selection and the lack of standardized collection criteria between the pandemic and post-pandemic periods may have introduced selection bias, potentially limiting the comparability of the two periods. Third, the PowerCheck Respiratory Virus Panel and the Real-Q Direct SARS-CoV-2 Detection Kit use different Ct cut-off values (34 and 38, respectively), which may have contributed to discrepancies between the two assays. Fourth, the efficacy of the PowerCheck Respiratory Virus Panel for identifying respiratory viruses other than SARS-CoV-2 was not assessed. Lastly, this study did not assess the clinical outcomes of codetected patients, which warrants further investigation in future studies. Thus, additional validation using diverse clinical specimens and reference nucleic acids is necessary to evaluate the assay’s performance for detecting other respiratory viruses, which is essential before establishing its broader diagnostic utility.
In conclusion, this study demonstrated that the PowerCheck Respiratory Virus Panel was effective in detecting SARS-CoV-2. Additionally, it highlighted the potential utility of respiratory virus multiplex real-time PCR during the post-pandemic period. As the prevalence of non–SARS-CoV-2 respiratory viral infections has increased in the post-pandemic period, multiplex real-time PCR assays that target a broad range of respiratory viruses—including SARS-CoV-2—may prove more beneficial than assays that target only SARS-CoV-2.
Notes
Acknowledgements
None.
Ethics approval
This study was approved by the Institutional Review Board of Keimyung University Dongsan Medical Center (2024-01-018) and was conducted in accordance with the principles of the Declaration of Helsinki.
Conflict of interest
The authors have nothing to disclose.
Funding
None.