AWS Storage Gateway & DataSync — Overview
Hybrid storage and data transfer is a core SAA-C03 topic appearing in 73+ questions. The key decision is: Storage Gateway (on-premises access to cloud storage) vs DataSync (one-time or scheduled data migration/sync). This post covers all gateway types, DataSync patterns, Snow Family decisions, and the Transfer Family for protocol-based access.
NFS/SMB/iSCSI interface
Local cache + S3/EBS backend
Agent-based transfer
NFS/SMB/HDFS → S3/EFS/FSx
Snowball Edge (80TB)
Snowcone (14TB)
Snowmobile (100PB)
Managed SFTP/FTPS/FTP
Direct to S3/EFS
Storage Gateway Types
| Type | Protocol | Backend | Use Case |
|---|---|---|---|
| S3 File Gateway | NFS, SMB | S3 (each file = S3 object) | On-prem apps accessing S3 via file share. Data lake ingestion. Backup to S3. |
| FSx File Gateway | SMB | FSx for Windows | Windows file share with local cache. Branch office access to central FSx. |
| Volume Gateway (Cached) | iSCSI | S3 (hot data cached locally) | Block storage for apps. Primary data in S3, frequently accessed cached locally. |
| Volume Gateway (Stored) | iSCSI | Local (async backup to S3) | Full dataset on-prem, asynchronous EBS snapshots to S3 for DR. |
| Tape Gateway | iSCSI (VTL) | S3 → Glacier/Deep Archive | Replace physical tape libraries. Backup software unchanged. |
Storage Gateway vs DataSync — Decision
| Feature | Storage Gateway | DataSync |
|---|---|---|
| Purpose | Ongoing hybrid access (on-prem apps use cloud storage) | Data movement/migration (copy data from A to B) |
| Access pattern | Continuous read/write from on-premises applications | One-time migration or scheduled sync (not continuous access) |
| Local cache | Yes (low-latency access to frequently used data) | No (agent transfers data, doesn’t provide access) |
| Bandwidth | Throttled to not saturate link | Maximizes bandwidth (10x faster than open-source tools) |
| Destinations | S3, FSx, EBS Snapshots, Glacier | S3, EFS, FSx (all types), Snow, between AWS storage |
| Exam answer | “On-prem app needs to read/write S3 as NFS” → File Gateway | “Migrate NAS to EFS” or “daily sync to S3” → DataSync |
DataSync — Key Features
- Agent: Deploy on-premises (VMware/Hyper-V/EC2) to connect to source storage
- Sources: NFS, SMB, HDFS, self-managed object storage, S3-compatible
- Destinations: S3 (all classes), EFS, FSx (Windows/Lustre/ONTAP/OpenZFS)
- Scheduling: One-time or recurring (hourly, daily, weekly)
- Incremental: Only transfers changed files (compares metadata)
- Bandwidth throttling: Configurable to avoid saturating network links
- Between AWS: Can sync between AWS services (S3 → EFS, cross-region S3 → S3)
- Verification: Data integrity verification after transfer
Snow Family — When Network Is Not Enough
| Device | Capacity | Use Case |
|---|---|---|
| Snowcone | 8-14 TB (HDD/SSD) | Edge computing, small data transfers, IoT collection |
| Snowball Edge Storage | 80 TB | Large data migration when network too slow. Compute available. |
| Snowball Edge Compute | 80 TB + GPU | Edge ML inference, video processing + data transfer |
| Snowmobile | 100 PB | Exabyte-scale data center migration (shipping container) |
Decision rule: Calculate transfer time. If network transfer takes > 1 week, use Snow Family. Formula: DataSize(GB) × 8 / BandwidthMbps / 86400 = days.
Transfer Family
- What: Managed SFTP, FTPS, FTP, and AS2 servers that store data directly in S3 or EFS
- Use case: Partners/vendors who require SFTP to send/receive files. No changes to their workflow.
- Authentication: AWS Directory Service, custom Lambda authorizer, or service-managed users
- Exam note: “Third-party partners upload files via SFTP to S3” → Transfer Family
Exam Tips
| Exam | Key Points |
|---|---|
| SAA-C03 | “On-prem app needs NFS access to S3” → S3 File Gateway. “Migrate file server to AWS” → DataSync → EFS/FSx. “Replace tape backups” → Tape Gateway. “80TB limited bandwidth” → Snowball Edge. “SFTP to S3” → Transfer Family. “Low-latency local access + cloud backup” → Volume Gateway Cached. “Full data on-prem + DR snapshots” → Volume Gateway Stored. “Daily sync NFS to S3” → DataSync with schedule. |
AWS Certification Exam Practice Questions
Question 1:
A company’s on-premises applications access files via NFS. They want to store this data in S3 for durability and cost savings, but applications must continue using NFS without modification. Which solution provides this?
- AWS DataSync to copy files to S3 on a schedule
- AWS S3 File Gateway presenting S3 as an NFS mount point with local caching
- Mount S3 directly using s3fs-fuse on Linux
- AWS Transfer Family with NFS protocol
Show Answer
Answer: B — S3 File Gateway provides an NFS/SMB interface that applications use without modification. Files written to the gateway are stored as S3 objects. A local cache provides low-latency access to recently used data. DataSync (A) copies data but doesn’t provide ongoing NFS access. s3fs (C) is unreliable and not recommended. Transfer Family (D) doesn’t support NFS.
Question 2:
A company needs to migrate 50TB from their on-premises NAS (NFS) to Amazon EFS. Their internet connection is 1 Gbps. They want the migration completed within 1 week with minimal disruption. Which approach is fastest?
- AWS Snowball Edge — ship device, load data, ship back
- AWS DataSync agent on-premises → EFS over Direct Connect or internet (transfers at near line speed)
- S3 File Gateway — let it sync over time
- rsync over SSH to an EC2 instance with EFS mounted
Show Answer
Answer: B — At 1 Gbps, 50TB takes approximately 4.6 days (50,000 GB × 8 / 1000 Mbps / 86400 = 4.6 days). This fits within 1 week over the network. DataSync maximizes bandwidth utilization (up to 10 Gbps), handles incremental transfer, and integrates directly with EFS. Snowball (A) takes 5-7 days for shipping alone. DataSync is the fastest when network bandwidth is sufficient.
Question 3:
A company’s backup software writes to a tape library using iSCSI VTL interface. They want to eliminate physical tapes and store backups in AWS at the lowest long-term cost. The backup software cannot be modified. Which solution fits?
- S3 File Gateway with lifecycle policy to Glacier
- Tape Gateway — presents virtual tapes via iSCSI VTL, archives to S3 Glacier Deep Archive
- DataSync scheduled backup to S3 Glacier
- AWS Backup with custom vault in Glacier
Show Answer
Answer: B — Tape Gateway is a virtual tape library (VTL) that presents iSCSI targets to backup software. The software sees virtual tapes — no modification needed. When tapes are ejected/archived, they move to S3 Glacier or Deep Archive (lowest cost). File Gateway (A) would require changing the backup software from tape to NFS. This is the standard tape replacement pattern.
Question 4:
A company needs to transfer 200TB of data to S3. Their internet bandwidth is 100 Mbps. Transfer would take approximately 185 days over the network. Which approach completes the transfer fastest?
- Provision AWS Direct Connect (10 Gbps) and use DataSync
- Order multiple Snowball Edge devices (80TB each), load in parallel, ship to AWS
- Use S3 Transfer Acceleration with multipart upload
- Increase internet bandwidth to 1 Gbps and use DataSync
Show Answer
Answer: B — At 100 Mbps, network transfer is impractical (185 days). Direct Connect (A) takes weeks to provision. Order 3 Snowball Edge devices (80TB each), load them in parallel (~2-3 days to load each), ship to AWS (~5-7 days). Total time: ~2 weeks vs 6+ months over network. This is the textbook “when to use Snow” scenario. Transfer Acceleration (C) still limited by 100 Mbps bandwidth.
Question 5:
A company’s partners upload daily files via SFTP. Currently this goes to an on-premises server. The company wants to move to AWS with minimal partner disruption — partners must continue using their existing SFTP clients and credentials. Files should land in S3. Which service provides this?
- EC2 instance running OpenSSH SFTP server with S3 mounted via s3fs
- AWS Transfer Family with SFTP protocol, configured to store files in S3
- S3 File Gateway with SFTP enabled
- API Gateway with a custom SFTP-to-S3 Lambda function
Show Answer
Answer: B — Transfer Family provides a fully managed SFTP server that stores files directly in S3. Partners connect to the same protocol (SFTP) with custom hostname. You can use existing SSH keys or passwords. No infrastructure to manage. File Gateway (C) doesn’t support SFTP protocol. EC2 SFTP (A) works but is self-managed (patching, scaling, HA).
Related Posts
- Hybrid Cloud Networking
- Migration Architecture (7Rs)
- Disaster Recovery Architecture
- Data Lake & Analytics Architecture
References
Frequently Asked Questions
Storage Gateway vs DataSync — which do I need?
Storage Gateway: When on-premises applications need ongoing access to cloud storage (NFS/SMB/iSCSI interface). It’s a persistent access point. DataSync: When you need to move or synchronize data between locations (migration, backup, replication). It’s a transfer tool. If apps need to read/write → Gateway. If you need to copy data → DataSync.
When should I use Snow Family vs DataSync?
Calculate network transfer time: DataSize(GB) × 8 ÷ BandwidthMbps ÷ 86400 = days. If transfer takes more than 1 week, consider Snow Family. Typical break-even: 10-50TB at 100 Mbps → borderline. 100TB+ at 100 Mbps → definitely Snow. Also consider: ongoing costs of higher bandwidth vs one-time Snow device rental.
What is the difference between Volume Gateway Cached vs Stored?
Cached: Primary data lives in S3, local cache stores frequently accessed data. Best when dataset is too large for local storage. Stored: Full dataset lives locally, asynchronous snapshots uploaded to S3. Best when you need low-latency access to entire dataset and S3 is just for backup/DR. Cached saves local storage; Stored provides better local performance.