News
Mystery deepens over missing files on NMRA cloud; service provider had no backup
The loss of an estimated two terabytes—or 2,000 gigabytes—of classified information from the Lanka Government Cloud (LGC) risks endangering the business relationships of local drugs companies with their foreign principals, industry sources warned this week.
The files comprised information pertaining to the formulation of drugs and other confidential supporting documents uploaded via the National Medicines Regulatory Authority (NMRA) portal.
The impact of the data loss is still unclear to private sector drugs companies, the sources said. The Sri Lanka Chamber of the Pharmaceutical Industry (SLCPI) alone has around 60 members, excluding smaller entities that also routinely seek NMRA registration for their medicines. Some are still in the process of notifying their principals that the files they submitted were “deleted”.
“They were massive files, very confidential, containing information such as how a drug is formulated given by the manufacturer to the local agent,” said one company representative. “And if I’m representing a manufacturer, I agree to maintain secrecy while the NMRA is also mandated by its Act to observe highest confidentiality with regards to information of this nature.”
The NMRA Act contains several clauses requiring its members, officers and employees to sign declarations “pledging to observe secrecy in respect of all matters connected with the affairs of the Authority”.
“If the data go into the wrong hands, foreign manufacturers can refuse to submit files to Sri Lankan companies and we will not receive pharmaceuticals for our healthcare,” the company source said. “That is dangerous. And the SLCPI is taking it up strongly.”
There is no indication, however, that the files were stolen. Information was sketchy as the case has been handed over to the Criminal Investigation Department. But Epic Technology Group–commonly known as Epic Lanka–which secured the contract from NMRA to digitalise drugs registration, has maintained that an employee deleted the data. The circumstances remain unclear.
“The uploading of files through the e-NMRA portal has now come to a grinding halt,” a representative of another company said. “We were told the information was not copied but deleted. We have no control whatsoever. The NMRA is the regulator and the service provider is Epic. We cannot directly complain to or liaise with Epic because we have no agreement with it.”
The registration portal on e-NMRA returns a “504 gateway timeout” error. EPIC Executive Chairman Nayan Dehigama did not answer text messages or the telephone.
The NMRA does not grant blanket registration to drugs. Each gets either a provisional two-year registration or a five-year full one. Applications for renewals and approvals must be lodged at least six months before the licences lapse.
“There is no request yet to re-submit the files and it’s not easy because they are large with each file taking considerable time to upload,” one of the company sources said. “Also, we are not sure whose data are missing, whose areavailable, none of that has been explained to us.”
The NMRA digitalisation project was implemented under the last administration in partnership with the Information and Communication Technology Agency (ICTA) of Sri Lanka. Epic Lanka was selected as the service provider after a tender process. It had been “insisted” that the data be stored in the LGC which is run by ICTA.
However, it is clear now that there has been no backup. According to the agreement between NMRA and Epic Lanka, the company is contractually obligated to provide a data-backup which is technically defined as “the practice of copying data from a primary to a secondary location, to protect it in case of a disaster, accident or malicious action”.
“Epic should have developed the backup,” an official source said, requesting anonymity. “This is where investigations are important.” He said it was not the LGC task to provide backup, that it merely gives the cloud infrastructure as a service.
“The technology service provider, if it is their product, is responsible for making backups because the cloud won’t do that,” he reiterated. “It is also best practice from an information technology perspective to do so. You can’t say you tried to update it and the data failed.”
What had happened was “very serious,” the official acknowledged, adding that it was crucial to know whether the act was committed intentionally or by mistake.
“The cloud is a large data centre, a cluster of servers,” another official said. He, too, did not wish to be named. “It has enough and more capacity to hold as many redundant data replicas as needed from service providers and clients. In this instance, the service provider did not fulfil its contractual obligations. And it would be hilarious for any blame to be apportioned to the LGC.”
Data redundancy occurs when the same piece of data is stored in two or more separate places. But it is not possible for ICTA (which runs LGC) to absolve itself totally of any responsibility. The memorandum of understanding signed between ICTA and any Government institution for LGC says that backup should be handled by the support and maintenance team of the respective organisation. But it also states: “ICTA shall ensure that data backup policies and security policies are properly implemented.”
Sanjana Hattotuwa, a PhD candidate at the University of Otago, New Zealand, feels ICTA’s argument is shallow in the modern context. A State-level cloud infrastructure platform, he says, should architecturally have mirroring built-in.
“To not have it is unacceptable and borders on the incredible,” he reflected. “You are talking about the inability or unwillingness at a State-level to do what users are encouraged and advised to do with their personal computers. So you can immediately understand the nature and magnitude of the problem, mismatch of priorities.”
“That aside, what ICTA says also brings up a number of other issues,” he continued. “If we take at face value what they say, that individual providers who use and leverage the LGC have to build their own mirroring and backup, then the question arises as to what policies and procedures there are for auditing technically the solutions that are then provided by individual or third-party service providers.”
“Take the case of NMRA and this debacle,” Mr Hattotuwa elaborated. “Before what happened had happened, was there an audit process before and after the tender was awarded around that specific need. If it is the case that the LGC doesn’t provide backup, then there must be a specific requirement and stringent auditing around that requirement because this is a basic when it concerns mission-critical data.”
“For the LGC not to have it, and for this to happen, and then for the third party providers to not have it implemented, fully implemented, or not even thought about, or for the implementation to be faulty, are all serious questions,” he emphasized.
What ICTA says of its LGC procedure, he added, was bizarre and “in no shape or form inspires trust in the cloud infrastructure”.
“Due diligence, due process, oversight, auditing, technical standards, tender procedures, accountability, trust, restoration of data and restitution of trust in LGC are all pertinent questions arising from this particular case,” Mr Hattotuwa asserted. “There are broader implications around the larger LGC cloud infrastructure as a consequence of just this one story.”