AWS storage & databases all 3 written

S3 is not a filesystem. RDS is not DynamoDB. Real uploads, real tables, and how to pick between them.

22 S3: objects, buckets, and why it is not a filesystem

A bucket is a flat namespace of keys, each pointing at a blob of bytes. reports/2026/january.csv is one single key that happens to contain slashes -- S3 never created a reports directory, and there is no way to list "just the directories" without imitating one by filtering key prefixes.

A key is not a path, even though it looks exactly like one

Keys with slashes: one flat string, not a nested path

ran at build time

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16aws s3 mb s3://course-storage-bucket > /dev/null

echo "hello" | aws s3 cp - s3://course-storage-bucket/reports/2026/january.csv > /dev/null
echo "world" | aws s3 cp - s3://course-storage-bucket/reports/2026/february.csv > /dev/null

echo "every key in the bucket, exactly as S3 stores it -- flat, not nested:"
aws s3api list-objects-v2 --bucket course-storage-bucket --query 'Contents[].Key' --output json

key_count=$(aws s3api list-objects-v2 --bucket course-storage-bucket --query 'length(Contents)' --output text)
echo
echo "there is no 'reports' object, and no 'reports/2026' object -- only the two full keys."
no_bare_prefix=$(aws s3api list-objects-v2 --bucket course-storage-bucket --query "Contents[?Key=='reports/2026'].Key" --output text)
echo "an object with the key 'reports/2026' by itself: '$no_bare_prefix' (there isn't one)"

check "both full keys were stored" "$key_count" "2"
check "no separate 'directory' object was ever created" "$no_bare_prefix" ""

What it printed, when the build ran it

every key in the bucket, exactly as S3 stores it -- flat, not nested:
[
    "reports/2026/february.csv",
    "reports/2026/january.csv"
]

there is no 'reports' object, and no 'reports/2026' object -- only the two full keys.
an object with the key 'reports/2026' by itself: '' (there isn't one)
ok   both full keys were stored
ok   no separate 'directory' object was ever created

What the console's "folder" view is actually doing. Both the AWS console and aws s3 ls (as opposed to s3api list-objects-v2) group keys by shared prefixes up to the next / and display that as a folder, purely for readability. It is a display trick over a flat list, not a real directory structure underneath.

Versioning: nothing is overwritten, every version is kept

With versioning on, uploading to the same key does not replace the old object -- it adds a new version and keeps the old one addressable, forever, until someone explicitly deletes that specific version.

Versioning: overwriting a key never deletes the previous content

ran at build time

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32aws s3 mb s3://course-versioned-bucket > /dev/null
aws s3api put-bucket-versioning --bucket course-versioned-bucket --versioning-configuration Status=Enabled

echo "version one" | aws s3 cp - s3://course-versioned-bucket/document.txt > /dev/null
echo "version two" | aws s3 cp - s3://course-versioned-bucket/document.txt > /dev/null
echo "version three" | aws s3 cp - s3://course-versioned-bucket/document.txt > /dev/null

echo "the CURRENT content, reading the key normally:"
aws s3 cp s3://course-versioned-bucket/document.txt -

echo
echo "every version S3 actually kept, oldest logic aside -- three separate, addressable objects:"
versions=$(aws s3api list-object-versions --bucket course-versioned-bucket --query 'Versions[].VersionId' --output json)
echo "$versions"

version_count=$(aws s3api list-object-versions --bucket course-versioned-bucket --query 'length(Versions)' --output text)

echo
echo "reading back the OLDEST version specifically, by its version ID, even though it is not current:"
oldest_id=$(aws s3api list-object-versions --bucket course-versioned-bucket \
  --query 'Versions[-1].VersionId' --output text)
# get-object writes the BODY to the file argument and a metadata summary to
# stdout separately -- redirect the metadata away and read the real file back,
# rather than letting the two run together into one captured string.
aws s3api get-object --bucket course-versioned-bucket --key document.txt \
  --version-id "$oldest_id" /tmp/cd-course-oldest.txt > /dev/null
oldest_content=$(cat /tmp/cd-course-oldest.txt)
rm -f /tmp/cd-course-oldest.txt
echo "$oldest_content"

check "three uploads to the same key produced three real, distinct versions" "$version_count" "3"
check "the oldest version is still fully readable, on demand, by ID" "$oldest_content" "version one"

What it printed, when the build ran it

the CURRENT content, reading the key normally:
version three

every version S3 actually kept, oldest logic aside -- three separate, addressable objects:
[
    "029b9a38-265d-47c2-9dc9-b7f32508ac0e",
    "ec1cb927-028a-4e2e-8815-b83c6be06390",
    "8b4e449a-fcf2-4439-a3c4-ba892693ea25"
]

reading back the OLDEST version specifically, by its version ID, even though it is not current:
version one
ok   three uploads to the same key produced three real, distinct versions
ok   the oldest version is still fully readable, on demand, by ID

Why this matters for cost, not just safety. Versioning is real protection against an accidental overwrite or a ransomware-style bulk delete -- but every kept version is billed storage, forever, unless a lifecycle rule cleans up old versions. Turning versioning on with no lifecycle policy is a common, quietly expensive mistake.

Lifecycle rules: a real, checkable policy, not a vague promise

A lifecycle rule is JSON, stored by S3, that you can read back and verify -- exactly like an IAM policy. "Old data expires automatically" is either a specific rule that exists, or it is not happening, and the only way to know which is to ask S3 directly.

A lifecycle rule, read back to confirm it is real

ran at build time

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27aws s3 mb s3://course-lifecycle-bucket > /dev/null

echo "before any rule exists, does this bucket expire anything automatically?"
before=$(aws s3api get-bucket-lifecycle-configuration --bucket course-lifecycle-bucket 2>&1)
before_exit=$?
echo "$before (exit $before_exit)"

echo
echo "adding a real rule: objects under logs/ expire after 30 days"
aws s3api put-bucket-lifecycle-configuration --bucket course-lifecycle-bucket --lifecycle-configuration '{
  "Rules": [{
    "ID": "expire-old-logs",
    "Filter": {"Prefix": "logs/"},
    "Status": "Enabled",
    "Expiration": {"Days": 30}
  }]
}'

after=$(aws s3api get-bucket-lifecycle-configuration --bucket course-lifecycle-bucket)
echo "$after"

check "before any rule, the configuration genuinely does not exist" \
  "$([ "$before_exit" -ne 0 ] && echo yes)" "yes"
check "the stored rule really does target the logs/ prefix" \
  "$(echo "$after" | grep -c '"Prefix": "logs/"')" "1"
check "the stored rule really does expire after 30 days" \
  "$(echo "$after" | grep -c '"Days": 30')" "1"

What it printed, when the build ran it

before any rule exists, does this bucket expire anything automatically?

An error occurred (NoSuchLifecycleConfiguration) when calling the GetBucketLifecycleConfiguration operation: The lifecycle configuration does not exist (exit 255)

adding a real rule: objects under logs/ expire after 30 days
{
    "TransitionDefaultMinimumObjectSize": "all_storage_classes_128K",
    "Rules": [
        {
            "Expiration": {
                "Days": 30
            },
            "ID": "expire-old-logs",
            "Filter": {
                "Prefix": "logs/"
            },
            "Status": "Enabled"
        }
    ]
}
ok   before any rule, the configuration genuinely does not exist
ok   the stored rule really does target the logs/ prefix
ok   the stored rule really does expire after 30 days

23 EBS vs S3 vs instance store: picking the right storage

S3 is an API you talk to over HTTP; EBS is a virtual disk attached to exactly one instance, in exactly one Availability Zone, that the OS sees as a real block device. They solve different problems, and picking the wrong one is not a style choice -- it produces a specific, predictable class of outage.

An EBS volume is pinned to the Availability Zone it was created in

A volume attaches successfully within its own zone

ran at build time

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22vol=$(aws ec2 create-volume --availability-zone us-east-1a --size 10 --volume-type gp3 --output json)
vol_id=$(echo "$vol" | grep -o '"VolumeId": "[^"]*"' | head -1 | cut -d'"' -f4)
vol_az=$(echo "$vol" | grep -o '"AvailabilityZone": "[^"]*"' | head -1 | cut -d'"' -f4)
echo "volume $vol_id created in $vol_az"

echo
echo "launching an instance in the SAME zone, and attaching -- this succeeds:"
instance=$(aws ec2 run-instances --image-id ami-03cf127a --instance-type t2.micro \
  --count 1 --placement AvailabilityZone="$vol_az" --output json)
id=$(echo "$instance" | grep -o '"InstanceId": "[^"]*"' | head -1 | cut -d'"' -f4)
instance_az=$(aws ec2 describe-instances --instance-ids "$id" \
  --query 'Reservations[0].Instances[0].Placement.AvailabilityZone' --output text)
echo "instance $id is in $instance_az"

attach_result=$(aws ec2 attach-volume --volume-id "$vol_id" --instance-id "$id" --device /dev/sdf 2>&1)
echo "$attach_result"

attached_id=$(aws ec2 describe-volumes --volume-ids "$vol_id" \
  --query 'Volumes[0].Attachments[0].InstanceId' --output text)

check "the volume and instance are confirmed to share one zone" "$vol_az" "$instance_az"
check "attaching within the same zone succeeds, and AWS records the attachment" "$attached_id" "$id"

What it printed, when the build ran it

volume vol-e7923e0690af22f1f created in us-east-1a

launching an instance in the SAME zone, and attaching -- this succeeds:
instance i-6d0329322dc286bd6 is in us-east-1a
{
    "VolumeId": "vol-e7923e0690af22f1f",
    "InstanceId": "i-6d0329322dc286bd6",
    "Device": "/dev/sdf",
    "State": "attaching",
    "AttachTime": "2026-09-15T20:18:58.000Z"
}
ok   the volume and instance are confirmed to share one zone
ok   attaching within the same zone succeeds, and AWS records the attachment

What real AWS does across a zone boundary, which this panel cannot demonstrate. On real EC2, attaching a volume to an instance in a different zone from where the volume was created fails outright, with InvalidVolumeZoneMismatch. The free local stand-in used throughout this course does not enforce that specific rule -- it would silently allow the mismatched attach -- so rather than let the page imply a check that didn't actually happen, this is stated plainly instead of run.

Why the zone boundary matters anyway. An EBS-backed instance cannot simply be relaunched in another zone after a failure and keep its disk -- the volume has to be snapshotted and restored, or the workload has to tolerate losing it. S3, by contrast, is regional: nothing in the S3 lesson ever needed to think about a zone at all.

What survives a stop, and what does not

An EBS volume is independent of the instance's own lifecycle -- stopping the instance does not touch the volume's data at all. This is checkable directly: write a file, stop the instance, and (conceptually, since moto does not model a guest OS) the volume's own AWS-tracked state persists across the stop untouched.

EBS survives a stop; it is independent of the instance's own state

ran at build time

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23vol=$(aws ec2 create-volume --availability-zone us-east-1a --size 10 --volume-type gp3 --output json)
vol_id=$(echo "$vol" | grep -o '"VolumeId": "[^"]*"' | head -1 | cut -d'"' -f4)

instance=$(aws ec2 run-instances --image-id ami-03cf127a --instance-type t2.micro \
  --count 1 --placement AvailabilityZone=us-east-1a --output json)
id=$(echo "$instance" | grep -o '"InstanceId": "[^"]*"' | head -1 | cut -d'"' -f4)

aws ec2 attach-volume --volume-id "$vol_id" --instance-id "$id" --device /dev/sdf > /dev/null

before_stop=$(aws ec2 describe-volumes --volume-ids "$vol_id" --query 'Volumes[0].State' --output text)
echo "volume state before stopping the instance: $before_stop"

aws ec2 stop-instances --instance-ids "$id" > /dev/null

after_stop=$(aws ec2 describe-volumes --volume-ids "$vol_id" --query 'Volumes[0].State' --output text)
echo "volume state after stopping the instance:  $after_stop"

still_attached=$(aws ec2 describe-volumes --volume-ids "$vol_id" \
  --query 'Volumes[0].Attachments[0].InstanceId' --output text)
echo "still attached to: $still_attached"

check "the volume remains in-use, unaffected by the instance stopping" "$after_stop" "in-use"
check "the attachment itself survives the stop" "$still_attached" "$id"

What it printed, when the build ran it

volume state before stopping the instance: in-use
volume state after stopping the instance:  in-use
still attached to: i-4fe76ce3fb3ea4745
ok   the volume remains in-use, unaffected by the instance stopping
ok   the attachment itself survives the stop

Instance store is the opposite, and this is the mistake it causes. An instance store volume is physically part of the host hardware, not attached network storage -- it is wiped on stop, and if AWS moves your instance to different hardware (which can happen on start), it is gone permanently. Nothing this course can query proves that directly (moto does not model instance-store semantics, and honestly, neither should a course pretend a stop is safe on hardware nobody controls) -- the practical rule is: never store anything you cannot regenerate on instance store, ever.

Sizing and cost: paying for provisioned capacity vs paying for what you use

EBS bills for the size you provisioned, whether you use it or not. S3 bills for actual bytes stored, no provisioning step at all. This is checkable directly: a 10 GB EBS volume with nothing written to it is still a 10 GB commitment; an S3 bucket with one small object in it costs for that object's actual size.

Provisioned capacity vs actual usage: two different billing shapes

ran at build time

1
2
3
4
5
6
7
8
9
10
11vol=$(aws ec2 create-volume --availability-zone us-east-1a --size 100 --volume-type gp3 --output json)
provisioned_size=$(echo "$vol" | grep -o '"Size": [0-9]*' | head -1 | grep -o '[0-9]*')
echo "provisioned an EBS volume of exactly ${provisioned_size} GB -- billed at this size regardless of use"

aws s3 mb s3://course-object-cost-bucket > /dev/null
echo "a small amount of actual data" | aws s3 cp - s3://course-object-cost-bucket/tiny-file.txt > /dev/null
object_size=$(aws s3api head-object --bucket course-object-cost-bucket --key tiny-file.txt --query 'ContentLength' --output text)
echo "stored an S3 object of exactly $object_size bytes -- billed at only that size"

check "the EBS volume is billed at its full provisioned size" "$provisioned_size" "100"
check "the S3 object is billed only for its actual, tiny content" "$([ "$object_size" -lt 100 ] && echo yes)" "yes"

What it printed, when the build ran it

provisioned an EBS volume of exactly 100 GB -- billed at this size regardless of use
stored an S3 object of exactly 30 bytes -- billed at only that size
ok   the EBS volume is billed at its full provisioned size
ok   the S3 object is billed only for its actual, tiny content

24 RDS vs DynamoDB: relational vs key-value

Choosing between RDS and DynamoDB is choosing between "I know my schema and want real relations and transactions" and "I want to write and read by key at any scale, and my item shapes are allowed to vary." Both statements are checkable, and this lesson checks them.

A real RDS instance -- provisioned and confirmed, not just requested

Real RDS provisioning takes several minutes even for the smallest instance class, because AWS is really standing up a managed database host behind the scenes. The free local stand-in used in this course fakes that instantly, which is a deliberate and useful difference to be honest about: this speed is not what a real provisioning experience feels like, and the lesson is what got configured, not how fast it happened.

A real RDS Postgres instance, configured and confirmed

ran at build time

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17aws rds create-db-instance --db-instance-identifier course-db --db-instance-class db.t3.micro \
  --engine postgres --master-username courseadmin --master-user-password coursepassword123 \
  --allocated-storage 20 --output json > /dev/null

status=$(aws rds describe-db-instances --db-instance-identifier course-db \
  --query 'DBInstances[0].DBInstanceStatus' --output text)
engine=$(aws rds describe-db-instances --db-instance-identifier course-db \
  --query 'DBInstances[0].Engine' --output text)
storage=$(aws rds describe-db-instances --db-instance-identifier course-db \
  --query 'DBInstances[0].AllocatedStorage' --output text)

echo "status: $status"
echo "engine: $engine"
echo "allocated storage: ${storage}GB"

check "the instance really provisioned the requested engine" "$engine" "postgres"
check "the instance really provisioned the requested storage size" "$storage" "20"

What it printed, when the build ran it

status: available
engine: postgres
allocated storage: 20GB
ok   the instance really provisioned the requested engine
ok   the instance really provisioned the requested storage size

Read that "status" line above with the right expectation. Real RDS reports creating for several minutes before available; the speed here is the local stand-in being honest about faking it instantly, not a claim about real AWS timing. Every other fact this panel checked -- the engine, the storage size -- is a real, structural fact about what was configured, which is the part that transfers directly to a real account.

DynamoDB: schemaless by design, checked with two genuinely different items

A relational table would reject (or silently null-fill) a row missing a column another row has. DynamoDB has no such concept beyond the key -- two items in the same table can have completely disjoint sets of attributes, and both are simply valid.

Two items, two different shapes, one table, zero errors

ran at build time

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23aws dynamodb create-table --table-name Students \
  --attribute-definitions AttributeName=StudentId,AttributeType=S \
  --key-schema AttributeName=StudentId,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST > /dev/null

echo "one item, with Name and Age:"
aws dynamodb put-item --table-name Students \
  --item '{"StudentId": {"S": "s1"}, "Name": {"S": "Ada"}, "Age": {"N": "22"}}'

echo "a SECOND item, in the same table, with a completely different set of attributes:"
aws dynamodb put-item --table-name Students \
  --item '{"StudentId": {"S": "s2"}, "Major": {"S": "Physics"}}'

echo
echo "scanning the whole table -- both shapes coexist with no error, no warning:"
scan=$(aws dynamodb scan --table-name Students --output json)
echo "$scan"

check "both items were accepted with no shape conflict" \
  "$(echo "$scan" | grep -c '"Count": 2')" "1"
check "the first item's Name attribute is present" "$(echo "$scan" | grep -c '"Ada"')" "1"
check "the second item's Major attribute is present, on a row with no Name at all" \
  "$(echo "$scan" | grep -c '"Physics"')" "1"

What it printed, when the build ran it

one item, with Name and Age:
a SECOND item, in the same table, with a completely different set of attributes:

scanning the whole table -- both shapes coexist with no error, no warning:
{
    "Items": [
        {
            "StudentId": {
                "S": "s1"
            },
            "Name": {
                "S": "Ada"
            },
            "Age": {
                "N": "22"
            }
        },
        {
            "StudentId": {
                "S": "s2"
            },
            "Major": {
                "S": "Physics"
            }
        }
    ],
    "Count": 2,
    "ScannedCount": 2,
    "ConsumedCapacity": null
}
ok   both items were accepted with no shape conflict
ok   the first item's Name attribute is present
ok   the second item's Major attribute is present, on a row with no Name at all

What this actually costs you, not just what it gives you. RDS gives up flexible item shapes but gains real joins, transactions across tables, and ad hoc SQL queries over relationships nobody anticipated in advance. DynamoDB gives up all of that but gains predictable, flat-rate performance at effectively unlimited scale, because every read and write is a key lookup, never a table scan across unpredictable joins.

Reading by key vs reading by "whatever you feel like asking"

DynamoDB is fast because every well-designed query hits a specific key. Asking it a question the key was not designed for costs a full table scan -- checkable directly by comparing a key-based read against the scan used above.

A key lookup vs a scan: DynamoDB's real performance boundary

ran at build time

1
2
3
4
5
6
7
8
9
10
11
12
13
14echo "a key-based read: exact, fast, and this is DynamoDB's entire design centre:"
get_result=$(aws dynamodb get-item --table-name Students --key '{"StudentId": {"S": "s1"}}' --output json)
echo "$get_result"

echo
echo "asking a question the key was never designed for -- 'find anyone majoring in Physics'"
echo "requires reading every item and checking each one: exactly what scan just did above."
scanned_count=$(aws dynamodb scan --table-name Students --output json --query 'ScannedCount')
echo "items scanned to find that one Physics student: $scanned_count"

check "the key-based read returned exactly the one item asked for" \
  "$(echo "$get_result" | grep -c '"s1"')" "1"
check "answering a non-key question required scanning every item in the table" \
  "$scanned_count" "2"

What it printed, when the build ran it

a key-based read: exact, fast, and this is DynamoDB's entire design centre:
{
    "Item": {
        "StudentId": {
            "S": "s1"
        },
        "Name": {
            "S": "Ada"
        },
        "Age": {
            "N": "22"
        }
    }
}

asking a question the key was never designed for -- 'find anyone majoring in Physics'
requires reading every item and checking each one: exactly what scan just did above.
items scanned to find that one Physics student: 2
ok   the key-based read returned exactly the one item asked for
ok   answering a non-key question required scanning every item in the table

This is the actual skill DynamoDB demands. Designing the partition key (and any secondary indexes) around the questions the application will actually ask is not optional tuning -- get it wrong, and every query becomes the scan just measured above, at whatever scale the table has grown to. RDS defers this problem to SQL's query planner; DynamoDB hands it to you, up front, at design time.

22 S3: objects, buckets, and why it is not a filesystem

A key is not a path, even though it looks exactly like one

Versioning: nothing is overwritten, every version is kept

Lifecycle rules: a real, checkable policy, not a vague promise

23 EBS vs S3 vs instance store: picking the right storage

An EBS volume is pinned to the Availability Zone it was created in

What survives a stop, and what does not

Sizing and cost: paying for provisioned capacity vs paying for what you use

24 RDS vs DynamoDB: relational vs key-value

A real RDS instance -- provisioned and confirmed, not just requested

DynamoDB: schemaless by design, checked with two genuinely different items

Reading by key vs reading by "whatever you feel like asking"

New writing, when it lands.