· Author: Third Wave Analytics

Achieving LIMS Implementation in Just Three Weeks for COVID Testing

An article in the June 18, 2020 edition of the journal Nature Biotechnology details a remarkable project at the  Innovative Genomics Institute at University of California, Berkeley (UCB): converting a research lab into a CLIA-certified COVID-19 testing lab in just three weeks. In this blog post, Bryan Warf, Third Wave Analytics Principal Scientist and project manager for the LIMS implementation, takes us behind the scenes to explain how Lockbox LIMS made this possible.

How long does it take to develop a Laboratory Information Management System (LIMS)? To manage human samples in a clinical testing lab, the process usually takes at least six to 12 months.

Unfortunately, the speed at which the 2020 coronavirus pandemic has spread requires every facet of our healthcare system to reevaluate “the usual” and, instead, understand what is actually possible.

This is the story of a collaboration, between Third Wave Analytics and the Innovative Genomics Institute (IGI) at the University of California, Berkeley, brought together to establish a pop-up COVID-19 testing lab in three weeks. The teams developed and launched the entire lab during a time frame that was unprecedented.

While Third Wave Analytics played a large role, we were just one piece of a bigger puzzle. The true owners of this story are the members of the clinical testing laboratory that used our software to create something no one had ever experienced before.

Sunday, March 15: The Emergency Request for a COVID-19 Testing Lab

In early March 2020, the San Francisco Bay Area experienced one of the first major outbreaks of the novel coronavirus in the U.S. Testing capacity for COVID-19 in the area was extremely limited. Without the ability to quickly test for the virus in a high throughput manner, the region would suffer a situation that could become incredibly dangerous.

On Sunday, March 15, Third Wave Analytics received an emergency email request from the IGI: Help create a pop-up COVID-19 testing laboratory for the Berkeley community with a LIMS software solution. The IGI had resources to create the testing lab, and were fully committed to doing anything possible to help their community.

The Bay Area was running out of time. The region needed testing capacity immediately.

The timeline for this emergency request was more aggressive than anything Third Wave had ever considered possible. IGI wanted to create this entirely new clinical testing lab, and start processing human samples, within three weeks. The creation of a new testing lab for clinical samples is always measured in years, at least one or more, by the time all the work is complete. The deadline was daunting.

Third Wave Analytics responded immediately. The request looked overwhelmingly difficult but we pledged to do whatever we could to help the laboratory management portion of the testing. Key personnel at Third Wave immediately cleared all activities. This project would require every minute of our waking hours.

Why is a LIMS important to this project?

What exactly is a LIMS used for? A LIMS has a variety of functions, but the most important is to track information about the samples you test in your lab.

Where is a particular sample in the process right now? What were the testing results? What reagents were used to test the sample? Who tested the sample, and where is it currently stored? A good LIMS can easily answer all those questions, and many others you might have.

Week 1, March 16-20: Defining Requirements for LIMS Implementation

The first step in implementing a LIMS is to define all the functions required by the people who will use it. A highly detailed understanding of the laboratory testing workflow is always required to define all the LIMS requirements; if requirements change significantly over time, this process can take months.

During the first week, Third Wave and IGI personnel met for dozens of hours to discuss the new lab’s future-state sample testing process. At every step in the anticipated workflow, participants asked the following four questions:

  1. What would the lab physically do with samples?
  2. How should samples be uniquely tracked at this step? (e.g. unique sample barcode, a unique plate barcode combined with a well ID)
  3. What information/data should the LIMS provide to help at this step? (e.g. auto-calculation of mastermix volumes)
  4. What information/data should the LIMS capture during this step? (e.g. concentration of the extracted RNA, the lot number of reagents used during testing)

The lab must consider the answers to these four questions for every step in their process, making requirements-gathering a time-consuming demand. It is very rare for a lab to have a full set of answers ready to go for their entire sample testing workflow. Extensive discussions are usually required to flesh out the information. Unfortunately, we did not have the luxury of time for extensive discussion and contemplation.

A team of experts—laboratory scientists, clinical sample testing experts, automation engineers, software developers, and executive management—was created to make all decisions, using the most up-to-date and accurate information available on the virus and its detection. This team was able to determine nearly all initial LIMS requirements within this first week.

The team took a “minimal viable product” (MVP) approach to ensure that only those features absolutely required to detect the virus in patients were included. Anything deemed a “nice-to-have” would be implemented at a later date.

Key requirements defined within the week:

  1. The clinical testing protocol would deviate from the FDA-approved Emergency Use Authorization (EUA) protocol; therefore, the test would be considered a Laboratory Developed Test (LDT). This would allow for unlimited flexibility in the testing process, but would require a more significant validation to be performed prior to launch.
  2. Samples would be uniquely barcoded using a predefined barcode schema. The laboratory would create all collection kits/tubes and provide them to collection sites. The lab would also generate a barcode to be included on each tube’s label. This gave the lab complete control over sample barcodes, preventing duplicate barcodes from being entered into the system.
  3. There would be two negative controls included prior to extraction (buffer only and normal Human RNA controls). There would also be two additional controls introduced directly prior to qPCR (positive and negative qPCR controls).
  4. The testing protocol would use reagents and qPCR instruments from Thermo Fisher Scientific to assess the presence of SARS-CoV-2 RNA.
  5. The RNA extraction protocol would include automation to create a 96-well plate, and then a manual process to extract RNA. This process would become fully automated at a later date.
  6. Each sample would have two replicate qPCR measurements performed for viral detection. qPCR plates would be manually created at first. At a later date, a new and fully automated 384-well plate methodology would be developed.
  7. The lab could potentially re-test a single sample up to only two additional times if there were failures during sample testing (e.g. a control failure).
  8. The raw CT data from samples would be imported into the LIMS, where the data would be fully analyzed to generate a final sample results.
      1. This would circumnavigate the additional software from Thermo Fisher that analyzes the raw data.
      2. A validated python script was required to reformat the data file generated by the qPCR instrument prior to import into the LIMS.

Third Wave Analytics translated all these requirements into a database model, and created an Entity-Relationship Diagram (ERD) to map how the records and data collected during testing would relate to each other. A variety of experts from both Third Wave and IGI reviewed and approved this model.

By the end of the first week, the team had created a preliminary system. It included the first draft of the process by which samples would be batched and updated throughout sample testing. Third Wave demonstrated this system to IGI to confirm all identified requirements were met appropriately.

This preliminary system included:

  • Sample accessioning into the system: sample record creation upon entry of a sample barcode
  • RNA extraction and qPCR protocol execution: an interface with which the lab could enter specific testing steps to be displayed to the user during execution of the two specific protocols
  • Automation for the batching of samples during both RNA extraction and qPCR
  • Automated sample tracking (e.g. automated sample status updating)
  • A highly automated sample re-testing process, during which samples could simply be scanned a second time to trigger a re-test record creation

Not included in this initial demo: Data auto-analysis, data review, and report generation/delivery

Week 2, March 21-28: COVID-19 Testing Lab LIMS Configuration and Initial User Testing

IGI enthusiastically received the demonstration at the end of the first week. Then, we began configuration of the entire system. The lab itself was also already working at a feverish pace: setting up instruments, finalizing sample testing workflow steps, and training volunteer scientists to test the clinical samples.

By the end of this second week, we completed LIMS configuration for the entire samples testing workflow. The LIMS functionality included:

  • Sample accessioning into the LIMS
  • Control sample entry into the LIMS
  • Sample/control batching for RNA extraction
  • RNA extraction protocol execution
  • Sample/control batching for qPCR (creation of two qPCR plates for each extraction plate)
  • qPCR protocol execution, including raw data upload

Third Wave and IGI personnel conducted extensive testing of the system throughout week 2. As the week continued, we made many refinements to the LIMS to address bugs or incorporate newly identified requirements, such as an updated control testing strategy and different qPCR instruments.

The Final Hurdle: Data Auto-Analysis

During the first two weeks, the implementation proceeded well with one very significant exception. Data auto-analysis was proving to be incredibly challenging. Multiple team members from Third Wave devoted close, in-depth attention to the task, but the effort was still falling behind schedule badly. The auto-analysis was slated for completion by the end of week 2. The design and testing of the auto-analysis process proved so difficult that it was not completed until near the end of the third week.

The difficulty stemmed from the 14 discrete steps required to generate a final result for any clinical sample. By the end of analysis for a single sample, 115 to 345 individual data points needed analysis, depending on whether the sample was re-tested. To add more complication, a large number of possible final sample result outcomes had to be evaluated and tested.

At a high level, the complex data auto-analysis steps included:

  1. Analysis of all extraction and qPCR controls
    1. Each batch includes eight control samples in total
    2. Each control sample contains eight raw data points and 11 analyzed data points
  2. Each control sample needs data pushed to all appropriate records
    1. Extraction control data must be pushed to the extraction plate
    2. qPCR control data must be pushed to the appropriate qPCR plate
    3. The overall QC pass/fail status of each plate (based upon all controls) must be calculated and then pushed to all clinical samples on those plates
  3. Calculation of the final QC status of a clinical sample
    1. The overall QC pass/fail status of a sample must be calculated based upon internal MS2 data in a sample, and the QC status of all three plates on which it was tested (one extraction plate and two qPCR plates
  4. Calculation of the result for the specific sample that was tested (e.g. initial sample testing, sample re-testing result)
    1. Each sample has eight raw data points and 18 analyzed data points, not including all data for the controls
    2. Five potential results could be generated: Positive, Negative, Inconclusive, Invalid, or Null (untested)
  5. Calculation of the final sample result, accounting for all sample level results (including the initial test result and all re-testing results)
    1. Up to three sample values must be compared to generate a final result (as any given sample could be tested up to three times)
    2. With five possible results for each sample, there are 125 potential combinations of three sample results
    3. Each of the 125 scenarios must be evaluated for clinical accuracy, with developers knowing the correct “expected” sample result for every possible scenario
    4. Analysis of each of the 125 scenarios had to be coded into the analysis, with extensive testing to ensure the expected final result was generated for that specific scenario

The most difficult challenge in configuring this auto-analysis was juggling the complex analysis requirements against the built-in system limitations that prevent any specific function from using too many system resources. (For instance, an exceptionally complicated formula field could use up too much CPU time.)

The time crunch required a variety of creative solutions to quickly produce a full analysis pipeline. The team relied on a combination of every available tool on the LIMS platform to create a fully automated analysis pipeline, which included dozens of formula fields, almost a dozen workflow rules, and multiple automated processes/flows.

Week 3, March 29-April 5: Testing, Debugging, and System Validation

The LIMS development was nearly complete by the end of week 2, but the system required extensive testing. We needed to ensure the LIMS could reliably handle the COVID-19 testing lab’s need to process 96 samples at once.

Testing that many samples simultaneously creates the potential for quite a lot of scenarios. We needed to confirm the system could handle as many “unexpected but possible” combinations that we could conceive. By week 3, we didn’t have much time left to perform all the testing and identify bugs that needed fixing.

The lab had been making excellent progress. The instrumentation was in place, the testing workflow completely locked down, and the analytical sample testing validation underway. IGI was on track for the launch window. The LIMS had to be ready too, or the testing lab would face delays.

Third Wave and the IGI lab personnel joined forces to perform an end-to-end system verification. The teams looked for bugs, errors, or unexpected outcomes. Any error found was fixed as quickly as possible, and re-tested immediately.

Tuesday, April 6: Launch of the COVID-19 Testing Lab

The timeline had seemed unfathomable. The tasks felt endless. The hours were long. But 22 days after UCB sent Third Wave that emergency request for help, their COVID-19 testing lab was ready to test clinical samples. On schedule.

Every part of the team had come together and delivered. The testing workflow was completely defined and codified in standard operating procedures (SOP). The testing instruments were in place and validated. The analytical validation demonstrated the test process could detect the insidious virus. Personnel were trained. The LIMS was developed, tested, and deployed.

UCB’s launch of their COVID-19 testing lab was full of emotion. We all felt relieved that this herculean task was complete and could help the community. We felt hopeful that we could overcome the virus together.

We also felt utterly exhausted. Everyone had sacrificed their physical and emotional health to achieve this goal. We all now shared the bond that comes with getting across the finish line together to achieve something that seemed unattainable only a few weeks earlier. We had done it as a cohesive team.

Wednesday, April 7: What Comes Next?

We felt deeply proud of launching the COVID-19 testing lab in just 22 days, but also knew the real work was just beginning. Tens of thousands of patients needed testing.

The speed of the launch required a narrowly focused MVP testing process and LIMS. There were countless moments when we said “We can implement that later.” We had now arrived at “later.”

Both the lab workflow and LIMS would benefit from endless improvements needed to increase throughput. We needed to develop tools for doctors to order tests for patients, and view results. We needed to facilitate results reporting to the California Department of Public Health. We needed to improve the lab’s overall operational efficiency.

At any point, it would have been easy to feel overwhelmed by the remaining tasks. But a little perspective really showed us where we would arrive: A fully functioning COVID-19 testing lab now exists. The lab could test patients’ samples, provide results, and hopefully save lives.

For the first time since the COVID-19 pandemic first crippled the Bay Area, we all felt just a little less scared.