Managed Service for Apache Flink Skill

Overview

This skill provides comprehensive guidance for developing, optimizing, and operating Apache Flink applications on AWS Managed Service for Apache Flink. It is designed to improve Apache Flink onboarding and improve operational excellence.

Disclaimer

All code generated by this skill is AI-assisted and should be thoroughly reviewed, tested, and validated by users before being used in production environments. AI-generated code may contain errors, security vulnerabilities, or suboptimal patterns that require human judgment to identify and correct. Any code changes adopted from this skill should be tagged with generated by Managed Service for Apache Flink Kiro Power in code comments or commit messages to clearly indicate when generated changes were adopted.

Key Problems Solved

1. New Customer Onboarding: Overcome the steep learning curve for Flink's Java-centric development model
2. Existing Customer Optimization: Systematic review and improvement of Flink applications to align to best practices
3. Operational Excellence: Integrated troubleshooting and monitoring using automated tools with CloudWatch and Managed Service for Apache Flink APIs
4. Migrating / upgrading to Flink 2: Systematic review and upgrade from Flink 1.x apps to Flink 2.x apps to handle migrations to 2.x with breaking changes

Reference Files

Load these files as needed based on the user's task:

• Core Managed Service for Apache Flink Development Workflow — how to approach user interaction, development, and testing
• Environment Setup — set up a local dev environment
• Dependency Management — Maven pom.xml dependencies for new projects
• Best Practices — Flink best practices for new apps
• Managed Service for Apache Flink Overview and Concepts — Managed Service for Apache Flink-specific optimizations and constraints
• Flink 2.x Migration Guide — upgrading from Flink 1.x to 2.x, including state compatibility
• Resource Optimization — KPU sizing, operator parallelism tuning, configuration overrides, and checkpoint resource impact
• Job Graph Architecture — job graph optimization, monolith anti-pattern, high fan-out, and operator-to-task-slot mapping
• Monitoring and Metrics — CloudWatch metrics analysis, recommended alarms, and custom metrics emission
• Logging Configuration — logging best practices, Log4j2 configuration, and rate-limited logging patterns for Managed Service for Apache Flink
• Kinesis Source / Sink Development — Kinesis connector-specific guidance

General Guidance

• Don't get started until you confirm the user's goals and use case
• Don't make changes unless the user asks you to
• When developing code, use this skill's reference files as guidance first, followed by general knowledge

CRITICAL: Required Reading for All Flink Development

BEFORE writing ANY Flink code, you MUST read these files:

0. ALWAYS read core-workflow.md first — Contains essential steps for development
1. ALWAYS read best-practices.md second — Contains essential Managed Service for Apache Flink patterns, anti-patterns, serialization guidance, state management, and configuration separation

These are NOT optional — you must read them before generating any code.

Working with Multiple Reference Files

When creating Flink applications, always reference multiple relevant files together:

1. Start with best-practices.md — Read this FIRST for all development work
2. Then read dependency-management.md — For correct Maven dependencies when building new Flink apps
3. Then read connector-specific guides (e.g., kinesis-connector-guide.md) for API usage whenever making changes to or evaluating app's source/sink configs or making new apps
4. For resource sizing and scaling — Read resource-optimization.md for KPU sizing, parallelism tuning, and checkpoint impact on resources
5. For job graph design — Read job-graph-architecture.md for operator chaining, data skew, anti-patterns, and task slot mapping
6. For monitoring and observability — Read monitoring-and-metrics.md for CloudWatch metrics analysis and custom metrics emission, and logging-configuration.md for log configuration
7. Cross-reference between documents — They link to each other for related information

Support & Legal

• Documentation: Amazon Managed Service for Apache Flink
• Support: GitHub Issues and AWS Support Center
• License: Apache-2.0
• Privacy Policy: AWS Privacy Notice