AWS re:Invent 2023 Announcements

AWS re:Invent 2023 Announcements

re:Invent always builds hype and excitement for the future direction of AWS. However, the noise at which features and functions are released can definitely lead to fatigue. I've compiled a list below of recent annoucements which I feel are very beneficial to AWS professionals.


Pattern grouping

Within CloudWatch Log Insights, all logs are now additionally grouped automatically via recognized patterns. Patterns are defined as simliar logs that only differ in some values. It's now much more straight forward to find alarming logs vs. traversing hundreds of log messages.

Anomaly detection

You can now enable anomaly detection on CloudWatch Log Groups. After a short time learning common patterns and variables for your Log Group. CloudWatch is capable of alerting on significant changes from increased log event counts to unexepected patterns.

Amazon CloudWatch Logs now offers automated pattern analytics and anomaly detection | Amazon Web Services
Searching through log data to find operational or business insights often feels like looking for a needle in a haystack. It usually requires you to manually filter and review individual log records. To help you with that, Amazon CloudWatch has added new capabilities to automatically recognize and cl…

Query generator

Generate AI with a proper use case? It appears like using natural language to build out CloudWatch queries is a welcome sight in most cases. You can now simply specify what you're searching for "find 5 longest Lambda invocation times" and get a prebuilt query which defines what you're looking for in SQL. Currently this is only available in us-east-1 and us-west-2 but it is free of charge!

Use natural language to query Amazon CloudWatch logs and metrics (preview) | Amazon Web Services
To make it easy to interact with your operational data, Amazon CloudWatch is introducing today natural language query generation for Logs and Metrics Insights. With this capability, powered by generative artificial intelligence (AI), you can describe in English the insights you are looking for, and…


Faster scaling

Scaling lambda and load testing have always been topics which linger in the back of one's mind. However, faster scaling looks to ease some of the ongoing worry. Prior to the announcement, Lambda could scale up between 500 and 3,000 new execution environments in a given minute which was region dependent. Now, it scales up by 1,000 environments every 10 seconds and each function is also scaled independently. Each Lambda function can create up to 1,000 environments every 10 seconds. There's still a limit of the account concurrency. The default quota for Lambda concurrent executions is 1,000 and even lower or new accounts. But you can request limit raises (like every other service limit) to tens of thousands if required.

Lambda scaling behavior - AWS Lambda
Learn how Lambda scales up your function to the concurrency quota for your AWS account.

Future runtime launch dates

AWS docs now include target dates for runtime rollouts. This ensures you're aware of what's coming down the pipe and can plan code updates accordingly. No more when Node X or Python Y type posts... YAY! Please clap for transparency.

Lambda runtimes - AWS Lambda
Learn about the languages that Lambda supports through runtimes.


direct s3 queries

You can now connect OpenSearch to S3 to make queries on the data objects which reside in your buckets. This is similar to Athena and leverages Glue Data Catalog to represent S3 data as tables. However, you can select various ingestions methods from only metadata to all data, which can help or hurt query response times. The thought here is to outperform Athena.

Announcing Amazon OpenSearch Service zero-ETL integration with Amazon S3 (preview) | Amazon Web Services
Today we are announcing a preview of Amazon OpenSearch Service zero-ETL integration with Amazon S3, a new way to query operational logs in Amazon S3 and S3-based data lakes without needing to switch between services. You can now analyze infrequently queried data in cloud object stores and simultaneo…


Express one zone storage class

Created to handle hundreds of thousands of requests per sec with consistent single-digit millisecond latency. Ideally this storage class is intended for working with small files and short life spans. Currently it's available within S3 via the AWS Console. This should be thought of as a cost effective trick for training AI/ML models for the time being.

Announcing the new Amazon S3 Express One Zone high performance storage class | Amazon Web Services
The new Amazon S3 Express One Zone storage class is designed to deliver up to 10x better performance than the S3 Standard storage class while handling hundreds of thousands of requests per second with consistent single-digit millisecond latency, making it a great fit for your most frequently accesse…

Step Functions

HTTP request task

This new task type allows you to make HTTP requests directly from a Step Function. This means there's no longer a need for a Lambda function to make an external API call. Its versatility stands out as a key advantage. Instead of being limited to integrating with specific API partners, the Step Functions team allows you to establish connections with nearly any API. You have the flexibility to configure the request method, body, headers, and query parameters, and you can even encode the body in the x-www-form-urlencoded form. While there are certain limitations, such as the requirement for the request body to be in JSON format and restrictions on certain headers (no XML or SOAP requests allowed), I find these constraints to be minor.

Redrive failed execution

Ever ran a step function which failed in the middle of an execution? Fear not! You can now redrive executions from a failed state. It's crucial to understand that the redrive will execute based on the exact Step Function definition, meaning you cannot alter the Step Function and rerun the failed step using an updated or corrected workflow version.

Test step

Wouldn’t it be great if you could run an individual state in order to test? The validation of Step Functions is time consuming enough. Test steps now allow you to ensure your functions operate as intended from a particular state. This allows you to debug ahead of time for things such as IAM permission errors.

External endpoints and testing of task states now available in AWS Step Functions | Amazon Web Services
Now AWS Step Functions HTTPS endpoints let you integrate third-party APIs and external services to your workflows. HTTPS endpoints provide a simpler way of making calls to external APIs and integrating with existing SaaS providers, like Stripe for handling payments, GitHub for code collaboration and…