AWS Startup
  • AWS 스타트업
  • AWS 스타트업 행사 일정표
  • AWS 스타트업 지원 프로그램
    • AWS Activate
    • Meet The Expert for Startup
      • Meet The Expert for TECH
      • Meet The Expert for BUSINESS
    • PaceMaker program for Startups
      • 고객 사례
  • AWS 스타트업 고객 사례
    • 보안/데이터베이스
    • DevOps/개발자도구/서버리스
    • 컨테이너
    • 데이터분석
      • Tappytoon 데이터 분석 파이프라인 구축기
    • AI/ML
      • Amazon SageMaker이용한 GI VITA의 MLOps 구축기
    • GenAI - Amazon Bedrock
    • SaaS
    • 글로벌 확장/비용 최적화
  • Startup Recipe
    • Architecture Recipes
      • 웹/애플리케이션 개발
      • 컨테이너
      • 데이터 분석
      • AI/ML
      • 생성형 AI
      • SaaS
    • Code Recipes
    • Demo Recipes
      • 생성형 AI
  • AWS 기본
    • AWS를 처음 시작한다면?
      • 1. 보안
      • 2. 네트워크
      • 3. 컴퓨팅
      • 4. 데이터베이스와 스토리지
      • 5. 비용 최적화
      • 6. 7 Effective AWS
  • AWS 보안
    • 보안
      • AWS 계정 안전하게 지키기 Tip
      • 멀티 팩터 인증(MFA)을 통해 AWS 계정을 안전하게 관리하기
      • 직원들의 사용자 계정에 MFA 적용하고 AWS 리소스 보호하기
      • AWS 계정 침해 사고 대응
      • Amazon GuardDuty를 이용한 침입 탐지 대응 전략
      • 아직도 SSH로 서버에 접속하니? 안전하게 서버 쉘에 접속하기!
      • 스타트업을 위한 보안 자가 진단 앱, Security Baseline Self Test
  • AWS 컨테이너
    • 📺AWS에서 시작하는 Container 생활
    • Amazon ECS
      • 📺Amazon ECS Service Connect 사용하기 | 기본편
      • 💻Hands-on Lab
    • Amazon EKS
      • 📺Amazon EKS를 통한 빠르고 편리한 컨테이너 플랫폼 활용
      • 📺Amazon EKS 마이그레이션 요점정리
      • 📺Amazon EKS로 간단한 웹 애플리케이션 구축하기
      • 📺Amazon EKS의 Observability 옵션들
      • 📺Amazon EKS의 Devops를 위한 Gitops 그리고 Progressive Delivery 소개
      • 💻Hands-on Lab
  • AWS 데이터 분석
    • AWS에서 데이터를 분석하는 방법
      • AWS에서 데이터 분석을 시작하기 위한 실시간, 배치 데이터 수집 방법 알아보기
      • 서버리스 데이터 분석
      • AWS 서비스를 이용하여 실시간 분석 시스템 구축하기
      • Lambda 아키텍처 데이터 분석 시스템 구축 하기
        • Part 1. 개념 및 워크 플로우
        • Part 2. 데모로 확인하기
    • AWS의 데이터 분석 서비스 소개
      • Amazon Quicksight로 파일 데이터 시각화해보기
      • Amazon Athena에 대해 알아보기
      • Amazon Kinesis Data Streams와 MSK를 비교해 보기
      • Amazon OpenSearch 업그레이드 및 Graviton2 사용하기
      • Amazon OpenSearch Service KNN 기능을 사용한 유사 이미지 검색 구현하기
    • 고객 사례
  • AWS 생성형 AI
    • 모든 스타트업을 위한 생성형 AI
    • RAG 아키텍처 - 개념부터 구현까지
  • AWS AI / ML
    • AI/ML
      • 한시간만에 AWS 머신러닝 서비스 따라잡기
      • 스타트업을 위한 AWS의 AI/ML 서비스 활용 방법 및 도입 전략
      • Amazon SageMaker로 Machine Learning 시작하기
      • Amazon SageMaker로 딥 러닝 기반 이미지 검색 서비스 만들기 - 개념 및 원리
      • Amazon SageMaker로 딥 러닝 기반 이미지 검색 서비스 만들기 - 구현 예제
      • Amazon Rekognition을 이용한 이미지 분석 및 검색 서비스 만들기
      • Amazon Rekognition Custom Labels를 이용한 나만의 이미지 분석 모델 만들기
      • Amazon Textact와 Amazon Neptune을 이용한 인맥 추천 서비스 만들기
      • Amazon SageMaker Canvas - a Visual, No-Code, AutoML tool for Business Analysts
      • Amazon SageMaker Model Deployment Strategies
      • JumpStart to Build Generative AI with Amazon SageMaker
    • Personalized Recommendations
      • 추천 시스템의 원리와 구축 사례
      • 5분만에 Amazon Personalize로 추천 시스템 구축하기
      • 추천 서비스를 위한 데이터 분석 시스템 구축하기
      • Amazon Personalize Recipes 120% 활용하기
  • AWS 비용최적화
    • 비용
      • 반드시 확인해야 할 비용 최적화 방법
      • 스타트업을 위한 6가지 AWS 비용 최적화 방법
      • 비용 및 리소스 관리를 위한 태그 생성 강제하기
Powered by GitBook
On this page
  • Build Business Intelligence System from Scratch on AWS
  • Zero-ETL integrations with Amazon Redshift
  • CDC(Change Data Capture) Data Pipeline
  • CDC(Change Data Capture) Data Pipeline using Amazon MSK and MSK Connect with Debezium
  • CDC(Change Data Capture) Data Pipeline using Amazon MSK Serverless and MSK Connect with Debezium
  • Transactional Data Lake supporting CDC-based Upsert operation
  • Transactional Data Lake using Amazon MSK and Apache Iceberg on AWS Glue
  • Transactional Data Lake using Amazon MSK Serverless and Apache Iceberg on AWS Glue
  • Transactional Data Lake using Amazon Data Firehose and Apache Iceberg
  • Streaming Data to Amazon S3 Tables using Amazon Kinesis Data Firehose
  • Streaming Data Pipeline from Apache Kafka to Amazon S3 using Amazon Kinesis Data Firehose
  • Redshift Streaming ingestion from Kinesis Data Streams, MSK, or MSK Serverelss (3 examples)
  • OpenSearch Serverless 4 Common Usage Patterns
  • Web Analytics System on AWS (a kind of Simple version of Google Analytics)
  • AWS Glue Streaming ETL example with Apache Iceberg
  • AWS Glue Streaming Ingestion from Kafka to Apache Iceberg table in S3
  • AWS Glue Streaming ETL example with Delta Lake
  • Building CQRS Pattern using Amazon Athena
  • Streaming Count Sketches with HyperLogLog in Amazon MemoryDB for Redis
  • Real-time Image Analysis System

Was this helpful?

  1. Startup Recipe
  2. Architecture Recipes

데이터 분석

Previous컨테이너NextAI/ML

Last updated 1 month ago

Was this helpful?

Build Business Intelligence System from Scratch on AWS

Describes the concepts of lambda architecture and the actual deployment process with an example of building a serverless business intelligence systems using Amazon Kinesis, S3, Athena, OpenSearch Service, and QuickSight.

Zero-ETL integrations with Amazon Redshift

(1) Aurora MySQL to Amazon Redshift

An Amazon Aurora MySQL zero-ETL integration with Amazon Redshift enables near real-time analytics and machine learning (ML) using Amazon Redshift on petabytes of transactional data from RDS.

(2) Aurora PostgreSQL to Amazon Redshift

An Amazon Aurora PostgreSQL zero-ETL integration with Amazon Redshift enables near real-time analytics and machine learning (ML) using Amazon Redshift on petabytes of transactional data from RDS.

(3) Amazon RDS MySQL to Amazon Redshift

An Amazon RDS MySQL zero-ETL integration with Amazon Redshift enables near real-time analytics and machine learning (ML) using Amazon Redshift on petabytes of transactional data from RDS.

CDC(Change Data Capture) Data Pipeline

Data Pipeline for CDC data from MySQL DB to Amazon OpenSearch Service through Amazon Kinesis using Amazon Data Migration Service(DMS).

CDC(Change Data Capture) Data Pipeline using Amazon MSK and MSK Connect with Debezium

Data Pipeline for CDC data from MySQL DB to Amazon S3 through Amazon MSK using Amazon MSK Connect (Debezium)

CDC(Change Data Capture) Data Pipeline using Amazon MSK Serverless and MSK Connect with Debezium

Data Pipeline for CDC data from MySQL DB to Amazon S3 through Amazon MSK Serverless using Amazon MSK Connect (Debezium)

Transactional Data Lake supporting CDC-based Upsert operation

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming and DMS

Transactional Data Lake using Amazon MSK and Apache Iceberg on AWS Glue

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming using Amazon MSK and MSK Connect (Debezium)

Transactional Data Lake using Amazon MSK Serverless and Apache Iceberg on AWS Glue

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming using Amazon MSK Serverless and MSK Connect (Debezium)

Transactional Data Lake using Amazon Data Firehose and Apache Iceberg

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with Amazon Data Firehose and DMS

Streaming Data to Amazon S3 Tables using Amazon Kinesis Data Firehose

This is a CDK Python project to build a fully managed data lake using Amazon Data Firehose and S3 Tables to store and analyze real-time streaming data.

Streaming Data Pipeline from Apache Kafka to Amazon S3 using Amazon Kinesis Data Firehose

Streaming data pipeline to continuously load data from an Amazon MSK or MSK Serverless cluster to Amazon S3 using Amazon Kinesis Data Firehose.

Redshift Streaming ingestion from Kinesis Data Streams, MSK, or MSK Serverelss (3 examples)

This is a collecton of CDK projects to show how to load data from streaming services into Amazon Redshift.

OpenSearch Serverless 4 Common Usage Patterns

Typical use cases of opensearch serverless: search, time-series, kinesis firehose integration, securing with VPC

  • (1) Search

  • (2) Time-series Log Analysis

  • (3) Streaming Ingestion through Kinesis Firehose

  • (4) Securing OpenSearch Serverless with VPC

Web Analytics System on AWS (a kind of Simple version of Google Analytics)

Web Log Analytics System with Parquet data format

This web analytics demo shows how to collect web logs with API Gateway and store them into S3 through Amazon Kinesis. Then this project shows how to analyze web logs with Amazon Athena.

Web Log Analytics System with Apache Iceberg Table

Web Log Analytics System using API Gateway integrated with Data Firehose with Apache Iceberg table

AWS Glue Streaming ETL example with Apache Iceberg

Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3

AWS Glue Streaming Ingestion from Kafka to Apache Iceberg table in S3

This is a collecton of Amazon CDK projects to show how to directly ingest streaming data from Amazon Mananged Service for Apache Kafka (MSK) and MSK Serverless into Apache Iceberg table in S3 with AWS Glue Streaming.

AWS Glue Streaming ETL example with Delta Lake

Streaming ETL job cases in AWS Glue to integrate Delta Lake and creating an in-place updatable data lake on Amazon S3

Building CQRS Pattern using Amazon Athena

Example of CQRS(Command and Query Responsibility Segregation) Pattern using Amazon Athena

Streaming Count Sketches with HyperLogLog in Amazon MemoryDB for Redis

This repository provides you cdk scripts and sample code on how to count unique items (e.g., unique visitors) with hyperloglog in Amazon MemoryDB for Redis. HyperLogLog (HLL) is a probabilistic data structure that estimates the cardinality of a set. As a probabilistic data structure, HyperLogLog trades perfect accuracy for efficient space utilization.

Real-time Image Analysis System

This sample project is a real-time image analysis system. As an image is uploaded, the real-time image analysis system annotates tags on the image using Amazon Rekognition and ingests image tags into Amazon Elasticsearch for analyzing image labels.

This repository provides you cdk scripts and sample code on how to implement a simple system. Below diagram shows what we are implementing.

This repository provides you cdk scripts and sample code on how to implement a simple system. Below diagram shows what we are implementing.

web analytics
web analytics
GitHub - aws-samples/aws-analytics-immersion-day: AWS Analytics Immersion Day - Build Business Intelligence System from ScratchGitHub
aws-kr-startup-samples/analytics/zero-etl-integrations/aurora-mysql-to-redshift at main · aws-samples/aws-kr-startup-samplesGitHub
aws-kr-startup-samples/analytics/zero-etl-integrations/aurora-postgresql-to-redshift at main · aws-samples/aws-kr-startup-samplesGitHub
aws-kr-startup-samples/analytics/zero-etl-integrations/rds-mysql-to-redshift at main · aws-samples/aws-kr-startup-samplesGitHub
Logo
GitHub - aws-samples/aws-dms-cdc-data-pipeline: Data Pipeline for CDC data from MySQL DB to Amazon OpenSearch Service through Amazon Kinesis using Amazon Data Migration Service(DMS).GitHub
GitHub - aws-samples/aws-msk-cdc-data-pipeline-with-debezium: Data Pipeline for CDC data from MySQL DB to Amazon S3 through Amazon MSK using Amazon MSK Connect (Debezium).GitHub
GitHub - aws-samples/aws-msk-serverless-cdc-data-pipeline-with-debezium: Data Pipeline for CDC data from MySQL DB to Amazon S3 through Amazon MSK Serverless using Amazon MSK Connect (Debezium).GitHub
GitHub - aws-samples/transactional-datalake-using-apache-iceberg-on-aws-glue: Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming and DMSGitHub
GitHub - aws-samples/transactional-datalake-using-amazon-msk-and-apache-iceberg-on-aws-glue: Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming and MSK Connect (Debezium)GitHub
GitHub - aws-samples/transactional-datalake-using-amazon-msk-serverless-and-apache-iceberg-on-aws-glue: Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming and MSK Connect (Debezium)GitHub
GitHub - aws-samples/transactional-datalake-using-amazon-datafirehose-iceberg: Stream CDC into an Amazon S3 data lake in Apache Iceberg table format with Amazon Data Firehose and DMSGitHub
aws-kr-startup-samples/analytics/streaming-data-to-s3tables-with-datafirehose at main · aws-samples/aws-kr-startup-samplesGitHub
GitHub - aws-samples/streaming-data-pipeline-from-kafka-to-s3-using-aws-kinesis-firehose: Streaming data pipeline to continuously load data from an Amazon MSK or MSK Serverless cluster to Amazon S3 using Amazon Kinesis Data Firehose.GitHub
GitHub - aws-samples/redshift-streaming-ingestion-patterns: This is a collecton of CDK projects to show how to load data from streaming services into Amazon Redshift.GitHub
GitHub - aws-samples/opensearch-serverless-common-usage-patterns: Typical use cases of opensearch serverelss: search, time-series, kinesis firehose integration, securing with VPCGitHub
Logo
Logo
Logo
Logo
GitHub - aws-samples/web-analytics-on-aws: This web analytics demo shows how to collect web logs with API Gateway and store them into S3 through Amazon Kinesis. Then this project shows how to analyze web logs with Amazon Athena.GitHub
GitHub - aws-samples/aws-glue-streaming-etl-with-apache-iceberg: Streaming ETL job cases in AWS Glue to integrate Iceberg and creating an in-place updatable data lake on Amazon S3GitHub
Logo
GitHub - aws-samples/aws-glue-streaming-ingestion-from-kafka-to-apache-iceberg: This is a collecton of Amazon CDK projects to show how to directly ingest streaming data from Amazon Mananged Service for Apache Kafka (MSK) and MSK Serverless into Apache Iceberg table in S3 with AWS Glue Streaming.GitHub
Logo
GitHub - aws-samples/aws-glue-streaming-etl-with-delta-lake: Streaming ETL job cases in AWS Glue to integrate Delta Lake and creating an in-place updatable data lake on Amazon S3GitHub
GitHub - aws-samples/aws-athena-cqrs-pattern: Example of CQRS(Command and Query Responsibility Segregation) Pattern using Amazon AthenaGitHub
Logo
Logo
Logo
GitHub - aws-samples/streaming-count-sketches-with-hyperloglog-in-amazon-memorydbGitHub
GitHub - aws-samples/aws-realtime-image-analysis: This sample project is a real-time image analysis system. As an image is uploaded, the real-time image analysis system annotates tags on the image using Amazon Rekognition and ingests image tags into Amazon Elasticsearch for analyzing image labels.GitHub
Logo
Logo
Logo
Logo
Logo
Logo
Logo
Logo
Logo
Logo
Logo
Logo