Amazon Web Services Launches New Services to Build and Deploy Faster, Manage Data Easier

Amazon Web Services is offering three new services to respond to increasing interest from AWS developers and enterprise IT users who want better ways to easily build, deploy and manage their data in the cloud. IDN looks at AWS Control Tower, AWS Security Hub, and AWS Lake Formation.

Tags: analytics, automation, AWS, cloud, data lake, devops, management, security, self-service,

AWS Security Hub

Amazon Web Services is offering three new services to respond to increasing interest from AWS developers and enterprise IT users who want better ways to easily build, deploy and manage their data in the cloud.


The three offerings -- AWS Control Tower, AWS Security Hub, and AWS Lake Formation – are intended in large part to fill some of the gaps some users see for managing a large number of accounts, applications and distributed teams, according to AWS officials.


In specific, AWS Control Tower, AWS Security Hub, and AWS Lake Formation, taken together will offer features to support provisioning and governance, monitoring security and compliance and to build and manage data lakes.  Further, the latest AWS services will help users automate processes, guide users in ways to build solutions more quickly, and more seamlessly work with added AWS services such as Amazon Lightsail, Amazon SageMaker machine learning and the  AWS Elastic Beanstalk environment to deploy and scale web services and apps.


 "We know our customers love the breadth of capability available in AWS, but they also tell us they want us to package our services in ways that make it easier for them to build architecture quickly,” said AWS Senior Vice President Charlie Bell in a statement.


“One of the central benefits of the cloud is that it removes the vast operational complexities of managing physical infrastructure. AWS’s new services abstract away additional complexity, speeding and simplifying the process of deploying and managing cloud workloads, so customers can build faster, operate more securely, and maintain consistent governance in a way that gives them more time to innovate,” Bell added.


To support ways to provide developers and IT ops more time to innovate, Amazon’s latest services will also automate multi-step processes, as well as centralize management of crucial elements of an AWS environment for better consistency and more visibility, he noted.


The Latest AWS Oferings in Action


AWS Security Hub is designed to be the central place where compliance and security are managed across the AWS environment.  The idea is to provide customers a way to see their compliance and security state in one view. 


In specific, AWS Security Hub provides ‘aggregate findings,” resulting in a comprehensive view of high-priority security alerts and compliance status across all AWS accounts.  AWS Security Hub organizes, protects and aggregates security alerts from multiple AWS services like Amazon GuardDuty, Amazon Macie, and other AWS partner solutions. Findings are summarized in a dashboard with tables and graphs.  


A little extra detail comes from the AWS website:

AWS Security Hub also reduces the effort of collecting and prioritizing security findings across accounts, from AWS services, and AWS partner tools.  It ingests data using a standard findings format, eliminating the need for time-consuming data conversion efforts. It then correlates findings across providers to prioritize the most important findings.

Based on these findings, AWS Security Hub lets users run automated, continuous account-level configuration and compliance checks based on industry standards and best practices, such as the Center for Internet Security (CIS) AWS Foundations Benchmark. These checks provide a compliance score and identify specific accounts and resources that require attention.

AWS Control Tower automates the setting up of a baseline or landing zone that is secure and compliant in a well-architected multi-account AWS environment.  The baseline is configured based on best practices that have been established by Amazon while working with both small and large enterprises to create a secure environment that makes it easier to monitor AWS workloads with rules for operations, security, and compliance.


With this in mind, with AWS Control Tower, distributed teams can provision new AWS account quickly. At the same time, the central teams know that such new accounts are created in line with centrally established policies.

For visibility, AWS Control Tower also provides an integrated dashboard for viewing a top-level summary of an AWS environment of all accounts in one place. Users can view details on the number of accounts provisioned, the number of policies enabled across your accounts, and the compliance status of those accounts.


The AWS website adds these details on Control Tower features and benefits:  

As enterprises migrate to AWS, they typically have a large number of applications and distributed teams. They often want to create multiple accounts to allow their teams to work independently, while still maintaining a consistent level of security and compliance. In addition, they use AWS’s management and security services, like AWS Organizations, AWS Service Catalog and AWS Config, that provide very granular controls over their workloads. They want to maintain this control, but they also want a way to centrally govern and enforce the best use of AWS services across all the accounts in their environment.

Control Tower automates the set-up of their landing zone and configures AWS management and security services based on established best practices in a secure, compliant, multi-account environment. Distributed teams are able to provision new AWS accounts quickly, while central teams have the peace of mind knowing that new accounts are aligned with centrally established, company-wide compliance policies. This gives you control over your environment, without sacrificing the speed and agility AWS provides your development teams.

AWS Lake Formation is a centralized and secure lake that stores all data in their original form for analysis. It makes it easy for customers to build a secure data lake by automating and simplifying all the complex processes that are usually required such as cleaning, collecting and cataloging data.  In fact, AWS noted that Lake Formation “makes it easy to set up a secure data lake in days.”


Interest in data lakes is increasing among AWS users, according to Amazon, “because they understand that it is useful in helping them have all their data in a central place where they can apply analytics with machine learning.” 


That said, managing a data lake today to capture these benefits has become a lot more complicated over the years. This can include many varied complicated (and often manual) tasks, including

  • monitoring data flows
  • setting up the partitions
  • matching like records
  • de-duplicating access to data sets
  • loading data from different sources
  • configuring access control setting
  • auditing access over time and
  • defining transformation jobs.


AWS Lake Formation users can “move, store, catalog, and clean. . . data faster. [S]imply point Lake Formation at data sources, and Lake Formation crawls those sources and moves the data into [a] new Amazon S3 data lake,” according to Amazon.


Further detail comes from the AWS website:

Creating a data lake with Lake Formation is as simple as defining where your data resides and what data access and security policies you want to apply. Lake Formation then collects and catalogs data from databases and object storage, moves the data into your new Amazon S3 data lake, cleans and classifies data using machine learning algorithms, and secures access to your sensitive data.

[U]sers can then access a centralized catalog of data which describes available data sets and their appropriate usage [and] leverage these data sets with their choice of analytics and machine learning services, like Amazon EMR for Apache Spark, Amazon Redshift, Amazon Athena, Amazon Sagemaker, and Amazon QuickSight. . .  Users can also combine these services without having to move data between silos.

In addition, AWS Lake Formation has built-in machine learning to deduplicate and find matching records (two entries that refer to the same thing) to increase data quality. It can also change data into formats like Apache Parquet and ORC for faster analytics, as well as help drive easier self-service access to data, Amazon added.


Previews for AWS Security Hub and AWS Control Tower are available now; preview for AWS Lake Formation is available “soon,” according to Amazon.