OpenSearch

Amazon OpenSearch

~ jayendrapatil

Amazon OpenSearch

  • Amazon OpenSearch Service is a managed service that makes it easy to deploy, operate, and scale OpenSearch clusters in the AWS Cloud.
  • is the successor to Elasticsearch Service and supports OpenSearch and legacy Elasticsearch OSS.
  • is a fully open-source search and analytics engine for use cases such as log analytics, real-time application monitoring, and clickstream analysis.
  • OpenSearch provides
    • instance types with numerous configurations of CPU, memory, and storage capacity, including cost-effective Graviton instances
    • Up to 3 PB of attached storage
    • Cost-effective UltraWarm and cold storage for read-only data
    • Integration with AWS IAM, VPC, VPC Security Groups
    • Encryption at Rest and in Transit
    • Authentication with Cognito, HTTP basic, or SAML authentication for OpenSearch Dashboards
    • Index-level, document-level, and field-level security
    • Multi-AZ setup with node allocation across two or three AZs in the same AWS Region
    • Dedicated master nodes to offload cluster management tasks
    • Automated snapshots to back up and restore OpenSearch Service domains
    • Integration with CloudWatch for monitoring, CloudTrail for auditing, S3, Kinesis, and DynamoDB for loading streaming data into OpenSearch Service.

OpenSearch Service Domain

  • An OpenSearch Service domain is synonymous with an OpenSearch cluster.
  • Domains are clusters with specified settings, instance types, instance counts, and storage resources.
  • automates common administrative tasks, such as performing backups, monitoring instances and patching software once the domain is running.
  • uses a blue/green deployment process when updating domains. Blue/green typically refers to the practice of running two production environments, one live and one idle, and switching the two as software changes are made.
  • All domains configured for multiple AZs have zone awareness enabled to ensure shards are distributed across AZs.

OpenSearch Security

  • OpenSearch Service domains support encryption at rest through AWS Key Management Service (KMS), node-to-node encryption over TLS, and the ability to require clients to communicate with HTTPS.
  • supports only symmetric encryption KMS keys, not asymmetric ones.
  • encrypts all indices, log files, swap files, and automated snapshots.
  • does not encrypt Manual snapshots and slow & error logs.
  • can be configured to be accessible with an endpoint within the VPC or a public endpoint accessible to the internet.
  • Network access for VPC endpoints is controlled by security groups and for public endpoints, access can be granted or restricted by IP address.
  • supports integration with Cognito, to allow the end-users to log-in to OpenSearch dashboards through enterprise identity providers such as Microsoft Active Directory using SAML 2.0, Cognito User Pools, and more.

OpenSearch Storage Tiers

  • OpenSearch Service supports three integrated storage tiers, Hot, UltraWarm and Cold.
  • Hot tier is powered by data nodes which are used for indexing, updating, and providing the fastest access to data.
  • UltraWarm nodes complement the hot tier by providing a fully managed, low-cost, read-only, warm storage tier for older and less-frequently accessed data.
  • UltraWarm uses S3 for storage and removes the need to configure a replica for the warm data.
  • Cold storage is a fully-managed lowest cost storage tier that makes it easy to securely store and analyze the historical logs on-demand.
  • Cold storage helps fully detach storage from compute when they are not actively performing analysis of their data and keep the data readily available at low cost.

OpenSearch Cross-Cluster Replication

  • Cross-cluster replication helps automate copying and synchronizing indices from one cluster to another at low latency in the same or different AWS Regions.
  • Domains participating in cross-cluster replications need to meet the following criteria:
    • Participating domains should be on Elasticsearch version 7.10
    • Participating domains need to have encryption in transit enabled
    • Participating domains need to have Fine-Grained Access Control (FGAC) enabled
    • Participating domains versions should adhere to the same rules as rolling version upgrade
  • Current implementation of cross-cluster replication does not support Ultrawarm or Cold Storage.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.

References

Amazon_OpenSearch