As in year’s past, Amazon Web Services CEO Andy Jassy formally kicked off annual AWS re:Invent user conference, being held in Las Vegas this week, by introducing a dizzying array of new cloud services, many taking direct aim at existing products and services in the marketplace.
First, though, he couldn’t help to take a dig at competitors, noting that “We are the only cloud provider who breaks out numbers in a clear way,” he said. According to AWS’ own estimates, AWS has a 51 percent market share, with a 46 percent annual growth on a $27 billion annual revenue run rate. The closest competitor is Microsoft Azure with about a 13 percent market share, and the Google Cloud Platform struggles along at about 3 percent. Other contenders don’t even have enough market share to break out, Jassy said snarkily, as a Larry Ellison-like face popped up on the illustrative chart behind him.
Overall, about 20 new services were introduced at the day’s events. Here are a few of the most notable new technologies that debuted, for those interested in scalable development and deployment:
Infrastructure
Outposts addresses the persistent market of the hybrid cloud, by offering a set of server racks that AWS will deliver and even install within the enterprise that can run AWS services, either through VMware VCloud or natively. Jassy noted that Outposts, a managed service, is designed to help enterprises with those on-prem applications that can’t easily be moved.
The “Control Tower” service manages identity and access management for an organization’s AWS usage, with best practice blueprints and “guardrails” or prepackaged compliance rules that can be automatically enforced.
Andy Jassy and VMware CEO Pat Gelsinger discuss Outposts at AWS Re:Invent 2018.
New File Systems: The Amazon FSx For Lustre offers a fully managed filesystem for compute-intensive workloads based on the HPC Lustre file system, with high throughput, low latency. Lustre has traditionally been used in high-performance computing workloads, though has a reputation for being difficult to set up. Jassy promises this managed service will be easy to use. Also, for the declining market of Windows Server users (Jassy’s description, AWS will offer Amazon FSx for Windows, which delivers native Windows File Server capability.
Ground-Station-as-a-Service actually announced the prior day, and represents a pretty radical extension of cloud computing into serving satellite-based communication and data collection. It consists of a set of ground station antennas around the globe, often collocated with AWS data centers. This service will provide, on a pay-per-use, the ability to upload and download satellite data
Data
With TimeStream, AWS has moved into the space of time-series databases. It is a serverless service, called TimeStream, that organizes data by time, with a separate processing engine, to ensure performance. This service could be ideal for the Internet of Things, edge computing, and system monitoring.
New Blockchain services: After sitting on the sideline of blockchain technologies for several years, watching for a business use the technology would solve, AWS has recognized a use case of offering a distributed ledger whose changes would be easily accessible to all parties involved. To this end, AWS offers QLDB, a fully managed ledger database with centralized authority. The company actually built this database for internal use, to log changes on the control plane). The company also introduced Amazon Managed Blockchain, which offers a fully managed versions of the Hyperledger and Ethereum frameworks.
AWS Lake Formation: Over 10,000 data lakes now run on AWS’ Simple Storage Service (S3), according to Jassy. Lake Formation can make the job of filling these data lakes easier. It can migrate different sources of data into the Lake, format the data and apply policy rules.
A new storage class, Glacier Deep Archive could serve as a potential replacement to tape, costing $1/TB, about a quarter the cost of standard Glacier, AWS existing cold-storage option. Data is retrievable within a few hours. This new service builds on the recently-announced S3 Intelligent Tiering, which uses machine learning to model data usage so that little-used data can be automatically offloaded to more economical storage options, and to move more consulted data up to faster options.
Machine Learning
SageMaker Updates: SageMaker is AWS’ managed service for running machine learning jobs, eliminating the need for hiring data scientists and infrastructure specialists to incorporate artificial intelligence into applications and services. Ground Truth is a service to automatically label data, which can help build models that would otherwise be too complicated to execute before. Also, the company launched a marketplace for machine learning models from third-party vendors. Also, performance improvements were touted. Jassy noted that machine learning use of the Tensorflow framework has had scaling challenges, with most users get only a 60 percent utilization on GPUs on average. AWS engineers have been able to push that to 90 percent for SageMaker DeepLearning services.
Amazon Elastic Inference will allow users to attach GPUs to any Elastic Cloud Compute (EC2) instance, offering 1-32TFlops on demand. It can auto-detect major machine learning framework, such as Tensorflow, Pytorch etc. Using this pooled GPU capability, can cut costs up to 75 percent in many cases, Jassy asserted. To further speed performance, the company has developed a latency-sensitive high-throughput processor for machine learning inferencing, called AWS Inferentia. Like Amazon Elastic Inference, Inferentia can be attached to EC2 instances and supports all major machine learning framework. It is also available with SageMaker.
Reinforcement Learning: To date, most of the commercialization around machine learning has been with supervised learning, which involves stepping through the training steps to create and refine a model of behavior. A bigger challenge in the field has been reinforcement learning, Jassy explained. “It totally changes how things can get done,” Jassy said of reinforcement learning. These types of jobs may not have one right answer, but rather a way to work through a complicated scenario (an example: playing Pac-Man, where there is no one way to win). SageMaker RL offers a set of tools to get started in this field. And, to help developers understand this new concept, the company unveiled AWS DeepRacer, the “world’s first autonomous racer car,” a 1/18th scale self-driving car that can be trained in SageMaker. The company plans to hold tournaments, starting at this conference and continuing to next year’s AWS Re:Invent, where different teams can build and compete their driving models.
We’ll follow up with more information to many of these technologies as time and reader interest dictates.
The post AWS Re:Invent Day 1: New Machine Learning, Data, Infrastructure Services appeared first on The New Stack.