AWS re:Invent 2023 Announcements
re:Invent always builds hype and excitement for the future direction of AWS. However, the noise at which features and functions are released can definitely lead to fatigue. I've compiled a list below of recent annoucements which I feel are very beneficial to AWS professionals.
CloudWatch
Pattern grouping
Within CloudWatch Log Insights, all logs are now additionally grouped automatically via recognized patterns. Patterns are defined as simliar logs that only differ in some values. It's now much more straight forward to find alarming logs vs. traversing hundreds of log messages.
Anomaly detection
You can now enable anomaly detection on CloudWatch Log Groups. After a short time learning common patterns and variables for your Log Group. CloudWatch is capable of alerting on significant changes from increased log event counts to unexepected patterns.
Danilo Poccia
Query generator
Generate AI with a proper use case? It appears like using natural language to build out CloudWatch queries is a welcome sight in most cases. You can now simply specify what you're searching for "find 5 longest Lambda invocation times" and get a prebuilt query which defines what you're looking for in SQL. Currently this is only available in us-east-1 and us-west-2 but it is free of charge!
Danilo Poccia
Lambda
Faster scaling
Scaling lambda and load testing have always been topics which linger in the back of one's mind. However, faster scaling looks to ease some of the ongoing worry. Prior to the announcement, Lambda could scale up between 500 and 3,000 new execution environments in a given minute which was region dependent. Now, it scales up by 1,000 environments every 10 seconds and each function is also scaled independently. Each Lambda function can create up to 1,000 environments every 10 seconds. There's still a limit of the account concurrency. The default quota for Lambda concurrent executions is 1,000 and even lower or new accounts. But you can request limit raises (like every other service limit) to tens of thousands if required.
Future runtime launch dates
AWS docs now include target dates for runtime rollouts. This ensures you're aware of what's coming down the pipe and can plan code updates accordingly. No more when Node X or Python Y type posts... YAY! Please clap for transparency.
OpenSearch
direct s3 queries
You can now connect OpenSearch to S3 to make queries on the data objects which reside in your buckets. This is similar to Athena and leverages Glue Data Catalog to represent S3 data as tables. However, you can select various ingestions methods from only metadata to all data, which can help or hurt query response times. The thought here is to outperform Athena.
Channy Yun
S3
Express one zone storage class
Created to handle hundreds of thousands of requests per sec with consistent single-digit millisecond latency. Ideally this storage class is intended for working with small files and short life spans. Currently it's available within S3 via the AWS Console. This should be thought of as a cost effective trick for training AI/ML models for the time being.
Jeff Barr
Step Functions
HTTP request task
This new task type allows you to make HTTP requests directly from a Step Function. This means there's no longer a need for a Lambda function to make an external API call. Its versatility stands out as a key advantage. Instead of being limited to integrating with specific API partners, the Step Functions team allows you to establish connections with nearly any API. You have the flexibility to configure the request method, body, headers, and query parameters, and you can even encode the body in the x-www-form-urlencoded form. While there are certain limitations, such as the requirement for the request body to be in JSON format and restrictions on certain headers (no XML or SOAP requests allowed), I find these constraints to be minor.
Redrive failed execution
Ever ran a step function which failed in the middle of an execution? Fear not! You can now redrive executions from a failed state. It's crucial to understand that the redrive will execute based on the exact Step Function definition, meaning you cannot alter the Step Function and rerun the failed step using an updated or corrected workflow version.
Test step
Wouldn’t it be great if you could run an individual state in order to test? The validation of Step Functions is time consuming enough. Test steps now allow you to ensure your functions operate as intended from a particular state. This allows you to debug ahead of time for things such as IAM permission errors.
Marcia Villalba