Using Big Data to Battle Coronavirus: Real-Life examples and Recommendations

SPsoft
6 min readNov 2, 2020

COVID-19 pandemic has shown that all technological advances still cannot protect us from diseases. However, using these advances can help monitor the coronavirus spreading, develop the vaccine faster, and advance the healthcare industry as a result. All the technological advances of the last decade have an important role in battling the pandemic.

Here is how cloud-based tools like Big Data, DevOps, Artificial Intelligence/Machine Learning, and Big Data analytics can be used to battle coronavirus.

Cloud computing infrastructure provides immense resources needed to store and manage vast amounts of data used in clinical studies. This helps shorten the time needed for pre-clinical and clinical trials and speed up the COVID-19 vaccine research. The latest cloud technologies provide scalable object storages, powerful query engines, robust security, and high availability features, along with logging and monitoring. All of these features are vital for ensuring stable infrastructure performance and successful completion of data analysis.

Big Data analytics platforms are used for gathering, transforming, and processing vast volumes of various data, from blood test results and MRI scans to vaccine test samples and pre-clinical trials data. All major cloud providers offer the building blocks for designing Big Data analytics systems of any complexity, so medical institutions have all the components they now need to look for the COVID-19 cure. Data transfer limitations that working with different networks impose make cloud datacenters a perfect choice for deploying Big Data analytics systems due to their high throughput capacity.

Artificial Intelligence (AI) algorithms and Machine Learning (ML) models are applied to quickly discover patterns in the results of this Big Data analysis. Once trained to diagnose some condition on images, AI algorithms can do it much better and faster than people, dramatically shortening the time needed to process millions of medical images.

However, these Big Data platforms are complex and consist of multiple parts that must be correctly configured in order to perform reliably. This is where DevOps and CI/CD tools come into action. DevOps approach helps automate infrastructure and software deployment and management, while Continuous Integration (CI) and Continuous Delivery (CD) tools enable automation of repetitive operations to ensure the resilience of system operations.

To further guarantee the stability and high performance of these Big Data analytics systems, cloud providers offer logging, monitoring, and alerting capabilities, ensuring the DevOps engineers stay on top of system management tasks and have visibility into the infrastructure performance.

Lastly, the data on the COVID-19 vaccine research is currently among the most valuable information, so ensuring its security is vital. Cloud computing providers offer the most advanced and sophisticated measures that ensure the security of this data.

Real-world examples of using Big Data to overcome COVID-19

Below are several examples of how Big Data technologies are used to help contain the COVID-19 pandemic outside of the medical field.

  1. Using mobile geolocation data to create COVID-19 spread heatmaps. Mobile carriers like Vodafone track the location of their customers to create heatmaps of the populace relocations. This helps identify potential outbreaks before they occur and take preventive measures, as well as monitoring the efficiency of lockdowns.
  2. Providing real-time updates on the pandemic spreading. Gisanddata platform aggregates Big Data from a variety of sources to provide real-time updates on the situation, both globally and per country.
  3. Adopting the “remote work” approach. The businesses need to continue working despite lockdowns and their reliance on cloud computing helps them overcome the challenges of working from home. When Big Data, as well as any other business-critical information, is processed in the cloud — there is no need for an expensive on-prem data center and the businesses can operate through the pandemic.
  4. Analyzing the updated customer behavior patterns. Lockdowns changed the patterns of our lives and businesses have to react to it to adjust their marketing strategies, supply chains, and customer support approaches. Using Big Data analytics helps understand the changing needs of your audience faster and react accordingly.
  5. Establishing foundations for efficient work of geographically dispersed teams. While many organizations prefer to work from the office, geographically dispersed teams will be a viable approach to workforce organization in the future. By using cloud computing to support remote work businesses lay the foundation for long-term post-pandemic sustainable growth. It is also invaluable for medics and scientists who can now better combine their efforts. This will undoubtedly lead to a boost in the efficiency of healthcare and pharmaceutical research in the future.

Concerns and Recommendations

Some of the examples of Big Data applications mentioned above raise certain concerns. CT or MRI scans are the patient’s personal information and should be handled in accordance with HIPAA requirements. tracking every individual using their GPS location is a violation of the right to privacy. Big Data analytics results depend heavily on the quality and size of the training data sets, which are just being gathered now and might be labeled incorrectly. Temporary regulations issued across the globe (like mandatory self-isolation for 2 weeks after crossing a border) violate the rights of the citizens, while seemingly doing quite little to stop the pandemic, etc.

These concerns should be addressed to ensure the population trusts the governments and aligns their efforts to battle the coronavirus. Here are some recommendations on how to overcome these concerns:

  • Using blockchain to ensure patient data security and privacy. The blockchain ledger is essentially a distributed, immutable, and decentralized database, where the validity of data is ensured by the consensus of the majority of the network nodes. Blockchain is already actively used in multiple healthcare solutions and platforms across the globe. Storing patient data in blockchain ledgers can ensure its safe and appropriate usage.
  • Using federated learning to speed up ML models training. AI algorithms are traditionally trained in the cloud and require huge computing resources to handle large centralized and labeled datasets. Federated learning (FL) uses distributed networks (like millions of apps on smartphones South Koreans use to inform the authorities of the results of self-health checks).
  • This way, the apps train primary modes using the data set on the patient’s phone, but instead of sending all the data to the central server, they send only the trained model. The central server in the cloud aggregates all the individual models and forms a new version of the central model, which is then sent back to the users. This way, ML model training takes much less time and resources.
  • Using incentives to improve the quality of Big Data analytics. As we mentioned before, the accuracy of the Big Data analysis depends greatly on the volume and accuracy of the model’s training data set. However, the size of COVID-19 data sets available to healthcare and government organizations is minuscule in comparison to the data stored by individuals, businesses, and organizations. If the majority of the population and companies are to be incentivized to share this data, this would dramatically improve the relevance and accuracy of COVID-19-related Big Data analytics.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _

The COVID-19 pandemic has reminded us of the fragility of our society and the importance of medical research to preserving life. The healthcare and biotechnology industries are on the rise, while Big Data and other innovative technologies developed within the last 2 decades are effectively used to help them reach the current objectives and secure future success.

--

--

SPsoft

SPsoft is a Managed Services Provider specializing in end-to-end software development: https://spsoft.com/