The PBS affiliate station is facing a predicament in retrieving historical archives

St. Louis Nine PBS is a local TV station under PBS, which has preserved over 70 years of radio and television materials, totaling approximately 50TB. These materials include reports on the 1993 flood, the COVID-19 pandemic and images of the local history of East St. Louis, totaling over 11,000 documents, most of which are historical materials that cannot be recreated. The TV station chose OSS as the cloud storage service provider, with OSS responsible for providing software, hardware and cloud storage services. OSS did not build its own data center but rented racks from Iron Mountain's Denver data center, deploying the servers storing the archives within the Iron Mountain facility. This is a typical multi-layer outsourcing model: the TV station connects to OSS, and OSS connects to the data center.

The contract between the two parties expired on March 6th. The TV station attempted to contact OSS for renewal in February but never received a response. According to the contract terms, after the contract expired, the TV station had 30 days to retrieve all the data. But when the deadline arrived, the TV station was directly cut off from access rights. Later inquiries revealed that OSS had already ceased operations, and its business status in Colorado had changed to default and invalid.

The TV station could only turn to Iron Mountain and voluntarily offered to bear all reasonable costs, hoping to directly retrieve the archives. Iron Mountain rejected this request. It's contract was signed with OSS, not Nine PBS. Iron Mountain stated that it did not have the authority to open the servers hosted by OSS and could not read the data inside. If unauthorized access was made to the third-party hardware, it would violate privacy terms and even leak data of other OSS customers.

After being unable to contact OSS, Nine PBS then filed a lawsuit against OSS in the St. Louis Circuit Court and obtained an judgment. The court confirmed the TV station owns the archives, and OSS must return or help migrate the data. But the judgment is only binding on OSS and can not directly compel Iron Mountain to release the servers. In the litigation, the judge first issued a temporary injunction that barred Iron Mountain from deleting or altering the data on the hard drive.

At the hearing, the judge issued a new decision. Iron Mountain is ordered to hand over the physical devices that contain the data. The TV station must find a third-party person familiar with the OSS system, giving priority to former OSS employees. They must complete the data extraction within 30 days. During the operation of this technician, they must not read or damage the data of other clients hosted by OSS in the same facility. Nine PBS has already contacted a former OSS employee who is willing to provide technical support. The court has also reserved a contingency plan. If there are complex situations such as encryption, the court will reconvene for discussion.

This case exposes a technical risk that is often overlooked in hosted cloud storage. Many enterprise users assume that the data they pay to store in the cloud can be downloaded at any time. However, under a multi-layer outsourcing architecture, control rights are divided layer by layer. The data logic of Nine PBS belongs to the TV station, but the physical carrier is the server purchased by OSS and placed in the Iron Mountain data center. The access accounts, permission systems, etc. are all managed by OSS. If OSS goes bankrupt, the account system will directly fail. Even if the hard drive is right in front of you, without the corresponding system permissions and decryption keys, it is very difficult to read the files inside.

This is largely different from directly renting a public cloud. Large public clouds like AWS have hardware belonging to the cloud provider, and users manage the data through accounts. The risk of service provider bankruptcy is relatively lower, and the data retrieval process will be clearly stated in the service agreement. While OSS belongs to a small cloud service provider that hosts hardware and packages it as cloud services for customers.

Encryption is the biggest unknown risk in this data recovery. If OSS encrypts the entire disk, the keys are only saved in the internal system of OSS. Even if the hard drive hardware is obtained, without the keys, the data still can not be decrypted. This is why the court reserved a second hearing. The value of third-party technicians is to be familiar with the system architecture of OSS, know the storage paths, authentication methods, and be able to safely export 50TB large-capacity video files. 50TB of video materials, in itself, have a huge migration workload. Continuous copying through a common gigabit network requires a long time. On-site extraction from the data center also needs to be done carefully to prevent file damage.