Man of quartz: Microsoft's Project Silica and 'saving' the world
Microsoft just had a breakthrough that could change the way we store data long-term, and how companies like Warner Bros protect the cultural legacy of over 100 years of entertainment.
At the 2019 Microsoft Ignite event in Orlando, Florida, the grandfather of technology space announced a slew of innovations and product offerings that are coming to its customers over the coming months.
The company continued to bet big on hybrid cloud with the announcement of Azure Arc, which will let enterprises use Microsoft Azure to manage resources across its competitors’ clouds; launched Project Cortex, an AI-driven knowledge network that is the company’s first commercial tool since Microsoft Teams; big updates to virtual assistant, Cortana; and more.
Sliding a little under the radar, perhaps, was an update on Project Silica, a collaborative endeavour between Microsoft and Warner Bros.
“Imagine if a title like the ‘Wizard of Oz’ or a show like ‘Friends’ wasn’t available for generation after generation to enjoy and see and understand,” said Warner Bros. Chief Technology Officer Vicky Colf in a recent company blog post. “We think that’s unimaginable, and that’s why we take the job of preserving and archiving our content extremely seriously.”
In pursuit of this goal, Microsoft and Warner Bros have managed to successfully store and retrieve a copy of the 1978 Superman movie on a square of glass the size of your average drinks coaster.
The quartz glass square, which can withstand being boiled in hot water, baked in an oven, microwaved, flooded, scoured, demagnetised and other environmental threats, contains 75.6GB of data.
Warner Bros approached Microsoft after getting wind of the tech giant’s ongoing experiments with ultrafast laser optics and AI. Warner Bros is the owner of the DC Cinematic Universe, New Line Cinema, Village Roadshow, and its own extensive roster of Warner Bros animation and live action production companies, “with the largest content library in the media and entertainment industry by many measures,” according to Colf. The corporation has been searching for years for ways to effectively safeguard its asset library. Everything from Casablanca to the original Looney Toons prints, is currently being stored either digitally or on film prints. For theatrical releases that are shot digitally, the company creates an archival third copy by converting it back to analog film which splits the footage into three color components - cyan, magenta and yellow - and transfers each onto black-and-white film negatives that won’t fade like colour film.
Now, Warner Bros is potentially looking at Project Silica to create a permanent physical asset to store important digital content and provide durable backup copies.
“When we shoot something digitally - with zeros and ones representing the pixels on the screen - and print that to an analog medium like film, you destroy the original pixel values. And, sure, it looks pretty good, but it’s not reversible,” said Brad Collar, Warner Bros. senior vice president of global archives and media engineering.
“If we can take the digital representation of those pixels and put it on a medium like silica and read it back off exactly as it was when it came out of the camera, we’ve done our preservation job to the very best of our ability. That’s what I love about this.”
“We are not trying to build things that you put in your house or play movies from. We are building storage that operates at the cloud scale,” said Ant Rowstron, partner deputy lab director of Microsoft Research Cambridge in the United Kingdom, which collaborated with University of Southampton to develop Project Silica.
“One big thing we wanted to eliminate is this expensive cycle of moving and rewriting data to the next generation. We really want something you can put on the shelf for 50 or 100 or 1,000 years and forget about until you need it,” he explains.
One of the major issues with mass cloud storage of data is the ongoing need to update and expand cloud capabilities. The drive behind the archival nature of Project Silica is that digital information can be transferred into a hyper-durable physical storage medium cheaply and at scale (something Warner Bros and Microsoft are confident looks likely) to be stuck in a vault and forgotten about.
The potential for this technology to also be used for things like medical and criminal record keeping is exciting to the engineers at Microsoft, particularly at a time when the amount of data in the world is exponentially increasing, as advances in storage capabilities start to see diminishing returns.
The decision to encode the world’s first quartz storage square with Superman: the movie felt very appropriate to the archivists at Warner Bros, who has previously discovered boxes of Superman radio serials recorded in the 1940s on record-sized pieces of glass.
“We actually found players that we could play these things back on, and they were just as good because they were stored on glass. And we were able to digitise and save those wonderful pieces of content,” Collar said.
“So now one of our oldest assets in our vault is glass and one of the newest technologies in our vault is glass. And they’re both Superman. So we really have come full circle,” he said.
Fastly's CDN Reportedly to Blame for Global Internet Outage
A huge outage has brought down a number of major websites around the world. Among those affected are gov.uk, Hulu, PayPal, Vimeo, and news outlets such as CNN, The Guardian, The New York Times, BBC, and Financial Times.
It is thought a glitch at Fastly ─ a popular CDN provider ─ is causing the worldwide issue. Fastly has confirmed it’s facing an outage on its status website but fails to specify a reason for the fault ─ only that the problem isn’t limited to a single data centre and, instead, is a “global CDN disruption” that is potentially affecting the company’s global network.
“We’re currently investigating potential impact to performance with our CDN services,” the firm said.
What is Fastly?
Fastly is a content delivery network (CDN) company that helps users view digital content more quickly. The company also provides security, video delivery, and so-called edge computing services. They use strategically distributed, highly performant POPs to help move data and applications closer to users and deliver up-to-date content quickly.
The firm has been proving increasingly popular among leading media websites. After going public on the New York Stock Exchange in 2019, shares rose exponentially in price, but after today’s outages, Fastly’s value has taken a sharp 5.21% fall and are currently trading at US$48.06.
What are CDNs?
Content delivery networks (CDNs) are a web of small computers, or servers, that link together to collaborate as a single computer. CDNs improve the performance of internet-connected devices by placing these servers as close as possible to the people using those devices in different locations, creating hundreds of points of presence, otherwise known as POPs.
They help minimise delays in loading web page content by reducing the physical distance between the server and the user. This helps users around the world view the same high-quality content without slow loading times.
Without a CDN, content origin servers must respond to every single end-user request. This results in significant traffic to the origin and subsequent load, thereby increasing the chances for origin failure if the traffic spikes are exceedingly high or if the load is persistent.
The Risk of CDNs
Over time, developers have attempted to protect users from the dangers of overreliance through the implementation of load balancing, DDoS (Denial of Service) protection, web application firewalls, and a myriad of other security features.
Clearly, by the state of today’s major website outage, these measures aren’t enough. Evidently, CDNs present a risk factor that is widely underestimated ─ which needs to be rectified with haste. Content delivery networks have become a key part of the global infrastructure, and so it’s imperative that organisations start to figure out risk mitigation strategies to protect companies reliant on the interconnected service from further disruption and disarray.