AWS Security Lake – Centralized Security Data Lake

• AWS Security Lake is a fully managed security data lake service that automatically centralizes security data from AWS environments, SaaS providers, on-premises, and cloud sources into a purpose-built data lake stored in your AWS account.
• Security Lake normalizes and consolidates security data into the Open Cybersecurity Schema Framework (OCSF) format, enabling faster threat detection, investigation, and response.
• The data lake is backed by Amazon S3 buckets, and you retain full ownership and control over your data.
• Security Lake uses Apache Iceberg open table format and Apache Parquet columnar storage for optimized query performance and cost-efficient storage.
• It integrates with AWS Organizations for multi-account management and supports cross-Region data aggregation through rollup Regions.
• Security Lake provides a subscriber model to grant third-party SIEM tools, analytics platforms, and custom applications access to the centralized security data.
• The service is highly relevant for the AWS Certified Security – Specialty (SCS-C02) exam, particularly in the domains of Security Logging and Monitoring, and Threat Detection and Incident Response.

Security Lake Architecture

• Security Lake creates a purpose-built security data lake in your account using Amazon S3 as the storage layer.
• Key architectural components include:
  • Data Collection Layer – Automatically collects log and event data from natively supported AWS services and allows ingestion of custom/third-party sources.
  • Data Normalization Layer – Converts all ingested data into OCSF format and Apache Parquet columnar format for consistency.
  • Storage Layer – Data stored in S3 buckets using Apache Iceberg table format, registered in AWS Glue Data Catalog.
  • Access Control Layer – AWS Lake Formation provides fine-grained access control to the data lake tables.
  • Subscriber Layer – Managed access for SIEM tools, analytics platforms, and custom consumers.

Open Cybersecurity Schema Framework (OCSF)

• OCSF is a collaborative, open-source schema framework developed by AWS and leading cybersecurity partners (including Splunk, IBM, and others).
• It provides a vendor-agnostic taxonomy for security events, eliminating the need for custom parsers for each data source.
• OCSF defines standardized event classes such as:
  • Network Activity (VPC Flow Logs, DNS queries)
  • API Activity (CloudTrail events)
  • Security Findings (GuardDuty, Security Hub)
  • Authentication events
  • File Activity and Process Activity
• By normalizing to OCSF, Security Lake enables correlation across disparate sources without custom ETL pipelines.
• Third-party sources must be converted to OCSF format before ingestion (no charge for third-party data ingestion).

Apache Iceberg Tables

• Security Lake stores data in Apache Iceberg open table format on top of Amazon S3.
• Iceberg provides:
  • ACID transactions – Ensures data consistency during concurrent writes.
  • Schema evolution – Supports adding/removing columns without rewriting data.
  • Partition evolution – Optimizes query patterns without data migration.
  • Time travel – Query historical snapshots of data.
  • Hidden partitioning – Automatic partition pruning for faster queries.
• Tables are registered in AWS Glue Data Catalog, enabling query access via Amazon Athena, Amazon Redshift Spectrum, and Amazon OpenSearch Service.
• The combination of Parquet format + Iceberg tables provides storage-efficient and query-optimized access to security data.

Data Lifecycle Management

• Security Lake provides customizable retention settings to manage data lifecycle.
• Supports automated storage tiering to transition data to cost-effective storage classes (S3 Standard → S3 Standard-IA → S3 Glacier).
• Data is automatically partitioned by source, Region, account, and time for efficient querying.
• Retention policies can be configured per source and per Region.

Security Lake Data Sources

• Security Lake supports natively integrated AWS sources and custom/third-party sources.
• AWS sources are automatically normalized to OCSF format and converted to Apache Parquet.

Natively Supported AWS Sources

Source	Data Type	OCSF Event Class
AWS CloudTrail (Management Events)	API calls, console sign-ins	API Activity
AWS CloudTrail (S3 Data Events)	S3 object-level operations	API Activity
AWS CloudTrail (Lambda Data Events)	Lambda invocation activity	API Activity
Amazon VPC Flow Logs	Network traffic metadata	Network Activity
Amazon Route 53 Resolver	DNS query logs	DNS Activity
Amazon S3 Data Events	Object access activity	API Activity
AWS Lambda	Execution activity	API Activity
Amazon EKS Audit Logs	Kubernetes API server audit logs	API Activity
AWS WAF	Web request logs	HTTP Activity
AWS Security Hub	Aggregated security findings	Security Finding

Security Hub Integrated Findings Sources

• Security Lake ingests findings from Security Hub, which aggregates from:
  • Amazon GuardDuty – Threat detection findings (malicious IPs, compromised instances, anomalous behavior)
  • Amazon Inspector – Vulnerability assessment findings
  • Amazon Macie – Sensitive data discovery findings
  • AWS Config – Configuration compliance findings
  • AWS Firewall Manager – Firewall policy compliance findings
  • IAM Access Analyzer – External access findings
  • AWS Health – Service health events
  • AWS Systems Manager Patch Manager – Patch compliance findings

Custom and Third-Party Sources

• Security Lake allows ingestion of custom sources from on-premises, other cloud providers, or SaaS applications.
• Custom sources must convert data to OCSF format before ingestion.
• There is no Security Lake charge for ingesting third-party or custom data (standard S3 storage charges apply).
• AWS AppFabric automatically normalizes SaaS application audit logs (Microsoft 365, Google Workspace, Salesforce, etc.) into OCSF and delivers to Security Lake.
• Custom sources can deliver data via:
  • Direct S3 writes to the Security Lake bucket
  • AWS Glue ETL pipelines for format conversion
  • Partner integrations (CrowdStrike, Palo Alto Networks, etc.)

Default vs. Optional Sources

• When enabling Security Lake, the default sources include: CloudTrail management events, VPC Flow Logs, Route 53 DNS logs, and Security Hub findings.
• CloudTrail S3 Data Events and AWS WAF logs are NOT included by default due to potentially high volume and cost – they must be explicitly enabled.
• EKS Audit Logs and Lambda Data Events are optional and can be enabled separately.

Security Lake Subscriber Model

• Security Lake uses a subscriber model to provide controlled access to the security data lake.
• Subscribers are external or internal consumers (SIEM tools, analytics platforms, custom applications) that need access to the centralized security data.
• Two types of subscriber access are available:

Data Access Subscribers

• Data access subscribers receive notifications when new objects are written to the data lake and can directly access the S3 objects.
• Subscribers are notified via:
  • HTTPS endpoint (default) – Security Lake sends notifications to a subscriber-provided endpoint.
  • Amazon SQS queue – Subscriber polls an SQS queue for new object notifications.
• Data access uses AWS Resource Access Manager (RAM) to share S3 resources cross-account.
• Best suited for SIEM tools that need to ingest raw data into their own systems (e.g., Splunk, Datadog).
• Subscriber specifies which log sources they are authorized to consume.

Query Access Subscribers

• Query access subscribers can directly query the data in place using AWS Lake Formation tables.
• Subscribers query data through services like Amazon Athena without needing to copy or move the data.
• Uses AWS Lake Formation permissions for fine-grained access control.
• Requires creation of an AmazonSecurityLakeMetaStoreManager role to manage AWS Glue partitions and table updates.
• Best suited for analytics tools that support federated queries or organizations that want to avoid data duplication.
• A query access subscriber can only query data in the AWS Region where it was created.

Subscriber Configuration

• When creating a subscriber, you specify:
  • Subscriber name and description
  • Log and event sources the subscriber can access
  • Data access method (S3 data access or Lake Formation query access)
  • Subscriber credentials (AWS account ID and external ID)
  • Notification method (SQS queue or subscription endpoint) for data access subscribers

SIEM Tool Integration

• Security Lake integrates with major SIEM and security analytics platforms through the subscriber model.
• OCSF normalization eliminates the need for custom parsers in each SIEM tool.

Splunk Integration

• Splunk integrates with Security Lake as a data access subscriber.
• Uses the Splunk Add-on for Amazon Security Lake to ingest OCSF-normalized data.
• Splunk polls SQS notifications and ingests new Parquet files from S3.
• Supports building SOC workflows with Security Lake as the centralized data source.
• Siemens uses this integration to centralize security data across 800+ AWS accounts.

Datadog Integration

• Datadog integrates with Security Lake for Cloud SIEM use cases.
• Datadog Observability Pipelines can send standardized OCSF-formatted logs to Security Lake.
• Supports bidirectional integration – ingest from Security Lake into Datadog Cloud SIEM, or route logs to Security Lake for long-term retention.
• Enables unified security monitoring across AWS and hybrid environments.

IBM QRadar Integration

• IBM QRadar integrates with Security Lake as a subscriber to ingest normalized security events.
• QRadar uses the Amazon Security Lake DSM (Device Support Module) to parse OCSF data.
• Enables centralized threat detection combining AWS security data with on-premises sources in QRadar.

Other Supported Integrations

• Amazon OpenSearch Service – Direct integration for interactive log analytics, real-time monitoring, pre-built OCSF dashboards, and on-demand indexing.
• Amazon Detective – Subscribes to Security Lake for enhanced investigation workflows.
• CrowdStrike – Sends endpoint detection data to Security Lake; consumes Security Lake data for correlation.
• Palo Alto Networks (Cortex XSIAM) – Ingests Security Lake data for extended detection and response.
• SentinelOne – Integrates for unified cloud and endpoint security analytics.
• Rapid7 – Consumes Security Lake data for managed detection and response.

Lake Formation Access Control

• AWS Lake Formation provides the access control layer for Security Lake data.
• Lake Formation manages permissions for the Glue Data Catalog tables that point to Security Lake S3 data.
• Key capabilities:
  • Table-level permissions – Control which subscribers can access which log source tables.
  • Column-level permissions – Restrict access to specific fields within a table (e.g., mask IP addresses).
  • Row-level security – Filter data based on attributes (e.g., only show data from specific accounts).
  • Cell-level security – Combine row and column filters for granular control.
• Lake Formation supports cross-account sharing via AWS Resource Access Manager (RAM), enabling subscribers in different accounts to query Security Lake data.
• The delegated administrator manages Lake Formation permissions for the organization’s security data lake.
• Lake Formation permissions are provided at no additional charge – you only pay for the underlying services (Glue, S3, Athena queries).

Cross-Account and Cross-Region Support

Multi-Account with AWS Organizations

• Security Lake integrates with AWS Organizations for centralized multi-account management.
• The Organizations management account designates a delegated Security Lake administrator.
• The delegated administrator can:
  • Enable Security Lake for all member accounts
  • Configure log sources for the entire organization
  • Automatically collect data from new organization accounts
  • Grant subscriber permissions to consume data from member accounts
• The management account cannot be set as the delegated administrator (security best practice).
• Each account sees its own usage on its bill, but consolidated billing applies through the organization management account.
• The delegated account is NOT billed for all accounts – each account incurs its own log collection charges.

Cross-Region with Rollup Regions

• Security Lake supports enabling across multiple AWS Regions simultaneously.
• Rollup Regions consolidate data from one or more contributing Regions into a central Region.
• Use cases for rollup Regions:
  • Compliance – Consolidate data into a Region that meets data residency requirements.
  • Centralized analysis – Query all security data from a single Region.
  • Reduced complexity – Subscribers only need to access one Region.
• Standard data transfer costs apply for cross-Region replication.
• Data in contributing Regions remains available locally – rollup creates a consolidated copy.

Security Lake Pricing

• Security Lake uses pay-as-you-go pricing with no upfront costs.
• Pricing is based on two dimensions:

Pricing Dimensions

Dimension	Description	Cost (US East)
CloudTrail Log Ingestion	Per GB of CloudTrail data (management + data events)	$0.75 per GB
Other AWS Log Ingestion	Per GB of VPC Flow Logs, Route 53, Security Hub, WAF, EKS, Lambda	$0.25 per GB
Data Normalization	Per GB for OCSF conversion and Parquet formatting	$0.035 per GB
Third-Party Data Ingestion	Custom/third-party source ingestion	$0 (no charge)
S3 Storage	Standard Amazon S3 storage charges	Standard S3 pricing
Lake Formation Permissions	Access control management	$0 (no charge)

Additional Costs

• AWS Glue – Orchestration costs for data catalog management.
• Amazon EventBridge – Event routing for subscriber notifications.
• AWS Lambda – Custom processing functions.
• Amazon SQS/SNS – Subscriber notification delivery.
• Amazon Athena – Per-query charges for query access subscribers.
• Data Transfer – Cross-Region replication for rollup Regions.

Cost Optimization

• Security Lake can offset existing costs by replacing duplicate CloudTrail trails or individual VPC Flow Log configurations.
• Use automated storage tiering to move older data to S3 Glacier.
• Carefully evaluate enabling CloudTrail S3 Data Events and WAF logs – these high-volume sources can significantly impact costs.
• A 15-day free trial is available with full feature access and a usage estimation tab in the console.

Security Lake vs. CloudWatch Logs vs. S3 Manual Aggregation

Feature	AWS Security Lake	CloudWatch Logs	S3 Manual Aggregation
Purpose	Centralized security data lake	Operational log monitoring	Custom data lake (DIY)
Data Normalization	Automatic OCSF normalization	No normalization (raw logs)	Manual ETL required
Schema	OCSF (standardized, open-source)	Source-specific formats	Custom schema (self-managed)
Storage Format	Apache Parquet + Iceberg tables	CloudWatch proprietary format	Any format (JSON, CSV, Parquet)
Multi-Account	Built-in via Organizations	Cross-account log subscriptions	Custom cross-account S3 policies
Multi-Region	Rollup Regions (built-in)	Per-Region (no native aggregation)	Custom S3 replication rules
Access Control	Lake Formation (fine-grained)	IAM policies + Log Groups	S3 bucket policies + IAM
Query Engine	Athena, OpenSearch, Redshift Spectrum	CloudWatch Logs Insights	Athena, custom tools
SIEM Integration	Native subscriber model	Log subscriptions to Kinesis/Lambda	Custom integration required
Retention Management	Automated tiering + custom retention	Log group retention policies	S3 Lifecycle policies (manual)
Setup Complexity	Low (managed service)	Low (per-service)	High (custom infrastructure)
Cost Model	Per-GB ingestion + normalization + S3	Per-GB ingestion + storage + queries	S3 + Glue + Athena + custom ETL
Best For	Security analytics, SIEM feeding, compliance	Real-time operational monitoring	Full customization, non-security data lakes

When to Use Each

• Security Lake – When you need centralized, normalized security data across multiple accounts/Regions for threat detection, incident response, and SIEM integration.
• CloudWatch Logs – When you need real-time operational monitoring, metric-based alarms, and log analysis for application debugging.
• S3 Manual Aggregation – When you have custom requirements, non-security data, or need complete control over the ETL pipeline and schema design.
• These solutions are complementary, not mutually exclusive – many organizations use CloudWatch Logs for real-time operations while using Security Lake for security analytics and long-term retention.

AWS Certification Relevance (SCS-C02)

• Security Lake is highly relevant for the AWS Certified Security – Specialty (SCS-C02) exam.
• Key exam domains where Security Lake appears:
  • Domain 2: Security Logging and Monitoring – Centralized log management, log aggregation across accounts/Regions, OCSF normalization.
  • Domain 1: Threat Detection and Incident Response – SIEM integration, threat detection workflows, subscriber model for security tools.
  • Domain 6: Management and Security Governance – Multi-account security management, Organizations integration, delegated administrator.
• Key concepts to understand for the exam:
  • How Security Lake differs from CloudWatch Logs and CloudTrail Lake
  • The subscriber model (data access vs. query access)
  • OCSF as the normalization standard
  • Cross-account and cross-Region data aggregation
  • Lake Formation for access control
  • Integration with Security Hub and GuardDuty

AWS Security Lake Practice Questions

1. A security team needs to centralize security logs from 50 AWS accounts across 4 Regions into a single data lake for their SIEM tool. The solution must normalize data from VPC Flow Logs, CloudTrail, GuardDuty findings, and third-party endpoint agents into a common schema. Which solution meets these requirements with the LEAST operational overhead?
   1. Create a centralized S3 bucket with cross-account bucket policies and use AWS Glue ETL jobs to normalize all log formats
   2. Enable Amazon Security Lake with AWS Organizations, configure rollup Regions, and use the OCSF schema for normalization
   3. Set up CloudWatch Logs cross-account subscriptions with Kinesis Data Firehose to deliver to a centralized S3 bucket
   4. Deploy AWS CloudTrail Lake with organization-wide event data stores in each Region
   Show Answer

   Answer: B –

   Explanation: Security Lake is purpose-built for this use case – it natively integrates with Organizations for multi-account collection, supports rollup Regions for cross-Region aggregation, automatically normalizes AWS sources to OCSF, and provides a subscriber model for SIEM tools. Third-party sources can also be ingested in OCSF format. Options A and C require significant custom engineering. Option D only covers CloudTrail events, not VPC Flow Logs or third-party sources.
2. A company uses Amazon Security Lake and wants their Splunk deployment to ingest new security events as soon as they arrive. The Splunk instance runs in a separate AWS account. Which subscriber configuration should be used?
   1. Create a query access subscriber and configure Splunk to run periodic Athena queries
   2. Create a data access subscriber with SQS queue notification and provide the Splunk account credentials
   3. Share the S3 bucket directly with the Splunk account using a bucket policy
   4. Configure CloudWatch Logs subscription filters to forward to the Splunk account
   Show Answer

   Answer: B –

   Explanation: For SIEM tools like Splunk that need to ingest raw data as it arrives, a data access subscriber with SQS notification is the correct approach. Splunk polls the SQS queue for new object notifications and ingests the Parquet files from S3. AWS Resource Access Manager handles cross-account S3 access. Option A (query access) is for in-place querying, not data ingestion. Options C and D bypass Security Lake’s managed subscriber model.
3. An organization enabled Security Lake but is concerned about costs. They notice that CloudTrail S3 Data Events and WAF logs account for 80% of their bill. What is the MOST cost-effective approach while maintaining security visibility?
   1. Disable Security Lake entirely and use CloudWatch Logs instead
   2. Remove CloudTrail S3 Data Events and WAF from Security Lake sources, and selectively enable them only for critical accounts or buckets
   3. Move all data to S3 Glacier immediately after ingestion
   4. Switch all subscribers from data access to query access
   Show Answer

   Answer: B –

   Explanation: CloudTrail S3 Data Events and WAF logs are high-volume sources that are NOT enabled by default in Security Lake for this exact reason. The recommended approach is to selectively enable these sources only where critical visibility is needed. Disabling Security Lake entirely (A) loses all centralized security data. Moving to Glacier immediately (C) makes data unavailable for real-time analysis. Subscriber type (D) doesn’t affect ingestion costs.
4. A security analyst with query access to Security Lake needs to investigate suspicious network traffic from a specific source IP across all Regions. Security Lake is enabled in 3 Regions with a rollup Region configured. How should the analyst run this investigation?
   1. Run separate Athena queries in each of the 3 contributing Regions
   2. Query the rollup Region using Amazon Athena, which contains consolidated data from all contributing Regions
   3. Use CloudWatch Logs Insights to search across all Regions simultaneously
   4. Create a new subscriber in each Region and aggregate results manually
   Show Answer

   Answer: B –

   Explanation: Rollup Regions consolidate data from contributing Regions into a central location. The analyst only needs to query the rollup Region using Athena to search across all Regional data. This eliminates the need for separate queries in each Region. Note that a query access subscriber can only query data in the Region where it was created, so querying the rollup Region provides the consolidated view.
5. A company wants to provide their managed security service provider (MSSP) access to Security Lake data. The MSSP needs to run custom threat hunting queries using Amazon Athena but should only see VPC Flow Logs and GuardDuty findings – not CloudTrail events. Which approach provides the required access with least privilege?
   1. Create a data access subscriber for the MSSP and filter notifications to only include VPC Flow Logs and GuardDuty
   2. Create a query access subscriber for the MSSP, specifying only VPC Flow Logs and Security Hub (GuardDuty findings) as authorized sources, with Lake Formation permissions
   3. Share the entire S3 bucket with the MSSP account and rely on IAM policies for filtering
   4. Export VPC Flow Logs and GuardDuty findings to a separate S3 bucket and share that with the MSSP
   Show Answer

   Answer: B –

   Explanation: A query access subscriber with specific source authorization is the correct approach. When creating a subscriber, you specify which log and event sources the subscriber can access. Lake Formation provides fine-grained access control to the specific tables. The MSSP uses Athena to query in place without data duplication. Option A (data access) would send raw data rather than enabling Athena queries. Options C and D bypass Security Lake’s managed access control model.

Frequently Asked Questions

What is AWS Security Lake?

AWS Security Lake automatically centralizes security data from AWS services, SaaS providers, and custom sources into a purpose-built data lake in your account. It normalizes data to the Open Cybersecurity Schema Framework (OCSF) and stores it as Apache Iceberg tables.

How does Security Lake differ from CloudWatch Logs?

CloudWatch Logs stores operational logs for monitoring and alerting. Security Lake normalizes security-specific data to OCSF format for long-term storage, cross-account aggregation, and integration with SIEM/analytics tools like Splunk and Datadog.

What data sources does Security Lake support?

Native sources include CloudTrail, VPC Flow Logs, Route 53 DNS, Security Hub findings, Lambda, EKS audit logs, WAF, and S3 data events. It also supports custom sources via OCSF-formatted Parquet files and third-party integrations.

Related Posts

• Amazon GuardDuty
• Amazon Detective
• AWS Security Services Cheat Sheet

References

• Amazon Security Lake Product Page
• Amazon Security Lake User Guide
• Amazon Security Lake Pricing
• Amazon Security Lake Features
• Security Lake Subscriber Management
• OCSF in Security Lake