AWS SDK 实战:S3 Express One Zone(Directory Bucket)基础入门场景全解析
AWS SDK 实战:S3 Express One Zone(Directory Bucket)基础入门场景全解析
本技术指南以 AWS 官方代码示例仓库中的 S3 Express One Zone 基础场景(Basics Scenario)为骨架,结合仓库内 PHP SDK 实现源码,完整讲解如何创建 VPC/VPC Endpoint、搭建最小权限 IAM 体系、创建 Directory Bucket 与普通 S3 Bucket、跨桶复制对象、用会话(Session)认证对比性能差异,以及观察两类 Bucket 在 ListObjects 排序上的本质区别。读完本文,你将掌握 S3 Express One Zone 的完整接入流程、关键配置参数、异常处理点与资源清理策略,并能在自己的 AWS 环境中复现这套可运行的基础示例。
场景概述:这份 SPECIFICATION 要演示什么
scenarios/basics/s3express/SPECIFICATION.md 定义的 SDK Basics 场景,面向对 S3 Express One Zone 尚不熟悉的开发者,目标是"开箱即用"地跑通以下核心交互:
- 创建 Directory Bucket(S3 Express 的核心载体);
- 使用会话(Session)换取更快的连接与鉴权;
- 在普通 Bucket 与 Directory Bucket 之间复制对象;
- 对比两类 Bucket 的下载性能与对象列举(
ListObjects)排序差异; - 演示结束后清理全部资源。
配套的概要文档 scenarios/basics/s3express/README.md 进一步列出该场景会调用的全部服务操作:CreateVpc、DescribeRouteTables、CreateVpcEndpoint(EC2),CreateBucket、CreateSession、PutObject、CopyObject、GetObject、ListObjectsV2、DeleteObject、DeleteBucket(S3),以及 DeleteVpcEndpoint、DeleteVpc(EC2 清理)。
仓库内以 PHP 实现为参考样例,核心文件为:
- 场景主程序:php/example_code/s3/express/S3ExpressBasics.php
- 独立运行入口:php/example_code/s3/express/ExpressRunner.php
- 底层 S3 服务封装:php/example_code/s3/S3Service.php
- IAM 资源模板:resources/cfn/s3_express_basics/s3_express_template.yml
- 集成测试:php/example_code/s3/express/tests/S3ExpressBasicsTest.php
运行前置条件
按 php/example_code/s3/express/S3ExpressBasics.php 文件头部的说明,运行本示例需要:
- 通过
composer install安装 AWS SDK for PHP v3 依赖; - 直接运行
php S3ExpressBasics.php,或以 PHPUnit 测试方式运行vendor/bin/phpunit S3ExpressBasicsTests.php; - 程序通过
ExpressRunner.php统一装配:require_once __DIR__ . "/vendor/autoload.php"后实例化S3ExpressBasics,依次调用helloService()与runExample(),最后在finally块中执行cleanUp(),确保即使中途抛错也会尽力清理资源。
需要注意:该场景会真实创建 AWS 资源(VPC、IAM 用户、S3 桶等),运行可能产生 AWS 账户费用;示例本身也建议遵循最小权限原则(least privilege)——这与场景第 2 步中"仅授予 S3 Express 所需权限"的设计目标一致。
场景执行流程(8 个步骤详解)
整体流程可划分为:网络准备 → 权限准备 → 双客户端 → 双桶 → 复制对象 → 性能对比 → 排序对比 → 清理。下面逐一结合源码展开。
第 1 步:搭建 VPC 与 VPC Endpoint
场景开始会向用户说明一个重要前提:S3 Express One Zone 的最佳实践是让计算资源与 Directory Bucket 位于同一可用区(AZ)。本地运行时的性能提升幅度会明显低于在 EC2 实例(与桶同 AZ)中运行。
程序随后提示用户选择运行环境:
$ec2Choice = testable_readline(
"Are you running this in an EC2 instance located in the same AZ as your intended Directory buckets? Enter Y/y to setup a VPC Endpoint, or N/n/blank to skip this section."
);
- 若回答
Y/y,程序使用 EC2 Client 依次执行:createVpc("10.0.0.0/16")创建 VPC,并记录VpcId;waitForVpcAvailable()等待 VPC 就绪;describeRouteTables()按vpc-id过滤获取该 VPC 的路由表;createVpcEndpoint($serviceName, $vpc['VpcId'], [$routeTable[0]])创建 VPC Endpoint。其中服务名按当前区域动态拼接为com.amazonaws.<region>.s3express——这正是 S3 Express 网关型 VPC Endpoint 的命名方式,官方推荐用它让 S3 Express 流量不经过 Internet Gateway 或 NAT 设备即可直达。
- 若回答其他值,则跳过该步骤,并提示
Skipping the VPC setup. Don't forget to use this in production!(本地演示可以跳过,生产环境不要忘记配置)。
规范文档特别注明:此步骤没有特有的异常需要单独处理(无 unique errors)。
第 2 步:用 CloudFormation 搭建 S3 Express 权限体系
这是本场景最关键的权限设计环节。程序不手写 IAM 资源,而是加载仓库内的 CloudFormation 模板 resources/cfn/s3_express_basics/s3_express_template.yml:
Resources:
RegularUser:
Type: AWS::IAM::User
ExpressUser:
Type: AWS::IAM::User
ExpressPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: AllowExpressOperations
PolicyDocument:
Statement:
- Effect: Allow
Action:
- "s3express:CreateBucket"
- "s3express:CreateSession"
Resource: "*"
Users:
- !Ref ExpressUser
Outputs:
RegularUser:
Value: !Ref RegularUser
ExpressUser:
Value: !Ref ExpressUser
模板设计要点:
- 创建两个用户:
RegularUser(普通 S3 操作)与ExpressUser(S3 Express 专用); ExpressUser仅被授予s3express:CreateBucket与s3express:CreateSession两个 Action,Resource 为*——这是最小权限的典型体现,S3 Express 场景尤其强调权限边界;- 通过 Stack Outputs(
RegularUser、ExpressUser)把用户名传给主程序。
PHP 侧执行逻辑(对应源码 S3ExpressBasics.php):
$this->cloudFormationClient->createStack([
'StackName' => $stackName,
'TemplateBody' => $file,
'Capabilities' => ['CAPABILITY_IAM'],
]);
- Stack 名称使用
"cfn-stack-s3-express-basics-" . uniqid()保证唯一; - 因为模板要创建 IAM 资源,必须显式声明
Capabilities => ['CAPABILITY_IAM']; - 随后用
getWaiter("StackCreateComplete")等待堆栈创建完成,并在CloudFormationException时输出getAwsErrorMessage()后重新抛出; - 堆栈创建成功后,程序读取 Outputs 拿到两个用户名,再调用
createAccessKey()为两个用户分别生成访问密钥,构建Aws\Credentials\Credentials对象。
规范文档要求此处检查 InvalidArgumentException(创建凭证/会话参数非法时的典型异常)。
第 3 步:创建两个 S3 Client
拿到两套凭证后,程序创建两个相互独立的 S3 客户端:
$s3RegularClient = new S3Client([
'Region' => $region,
'Credentials' => $regularCredentials,
]);
$s3ExpressClient = new S3Client([
'Region' => $region,
'Credentials' => $expressCredentials,
]);
- 普通客户端使用
RegularUser凭证,负责操作普通 S3 Bucket; - Express 客户端使用
ExpressUser凭证(携带AllowExpressOperations策略),负责操作 Directory Bucket。
源码注释强调:权限意识(adhering to least-privilege)对 S3 Express 至关重要。此步骤同样需要处理 InvalidArgumentException。
第 4 步:创建两个 Bucket
创建 Directory Bucket
Directory Bucket 是 S3 Express One Zone 的"枢纽"(linchpin),其命名有严格格式约束。源码中使用:
$directoryBucketName = "s3-express-demo-directory-bucket-$uuid--$az--x-s3";
命名由三部分组成:自定义前缀 + --<availability zone id>--x-s3 后缀。示例中区域为 us-west-2,可用区 ID 为 usw2-az1。
创建参数是理解 Directory Bucket 的关键:
$s3ExpressService->createBucket($directoryBucketName, [
'CreateBucketConfiguration' => [
'Bucket' => [
'Type' => "Directory", // 指定创建 Directory Bucket 而非普通桶
'DataRedundancy' => "SingleAvailabilityZone",
],
'Location' => [
'Name' => $az, // 可用区 ID
'Type' => "AvailabilityZone",
],
],
]);
要点:
Bucket.Type = "Directory"是让 S3 创建 Directory Bucket 的决定性参数;Bucket.DataRedundancy = "SingleAvailabilityZone"表明数据只冗余在单个 AZ(这是 S3 Express 与普通 S3 多 AZ 冗余的本质差异之一);Location.Name必须填可用区 ID(如usw2-az1),而不是可用区名称。
创建普通 Bucket
$normalBucketName = "normal-bucket-$uuid";
$s3RegularService->createBucket($normalBucketName);
普通桶命名只需全局唯一即可,无需 AZ 后缀。规范要求该步骤捕获 BucketAlreadyExists 异常(Bucket 名冲突时抛出)。
第 5 步:创建对象并跨桶复制
程序先在普通桶写入一个文本对象:
$objectKey = "basic-text-object";
$s3RegularService->putObject($normalBucketName, $objectKey,
['Body' => "Look Ma, I'm a bucket!"]);
然后先用 Express 客户端为 Directory Bucket 建立会话,再执行复制:
// 创建访问 directory bucket 的会话,SDK Client 会自动按需刷新
$s3ExpressService->createSession($directoryBucketName);
$s3ExpressService->copyObject($directoryBucketName, $objectKey,
"$normalBucketName/$objectKey");
createSession 在底层服务封装 S3Service.php 中实现:
public function createSession(string $bucketName)
{
$result = $this->client->createSession(['Bucket' => $bucketName]);
return $result;
// 捕获 S3Exception,若错误类型为 NoSuchBucket 则提示桶不存在
}
这里体现了 S3 Express 鉴权模型的核心差异:普通 Bucket 每次调用都会校验权限,而 Directory Bucket 利用用户凭证 + 会话令牌(session token)一次性校验。单次调用节省的时间不多,但在大量并发调用场景下累积起来非常可观。这也是场景把"CreateSession"作为独立演示环节的原因。
源码注释还特别指出一个易错点:Copy 操作对 Directory Bucket 并不受限("Copy operations are not restricted for Directory buckets"),但用户权限仍然是必须时刻记住的前提。规范文档对这一步的异常处理说明是"无相关特有错误"。
第 6 步:演示性能差异
程序对同一对象分别从两个桶各下载 X 次(默认 X = 1000),并用 hrtime(true) 高精度计时:
$downloads = 1000;
// 允许用户自定义下载次数(< 1000000)
$directoryStartTime = hrtime(true);
for ($i = 0; $i < $downloads; ++$i) {
$s3ExpressService->getObject($directoryBucketName, $objectKey);
}
$directoryTimeDiff = hrtime(true) - $directoryStartTime;
// 同样的循环对普通桶计时……
输出形如:
The directory bucket took 2464126625 nanoseconds, while the normal bucket took 1989507125.
That's a difference of -474619500 nanoseconds, or -0.4746195 seconds.
需要强调的是:该数值只是单次运行的样例输出,并非稳定性能结论。规范文档明确提示:性能差异只有在 EC2 实例与桶位于同一 AZ 时才会显著放大;本地运行时差异会小得多。实际数值取决于网络环境、并发方式与运行位置。规范要求此步骤捕获 NoSuchKey 异常(对象不存在时抛出)。如果所用语言支持并发,规范建议并行执行下载以更接近真实高并发场景。
第 7 步:填充对象,观察字典序差异
程序向两个桶分别写入 3 个带层级路径的对象:
$otherObject = "other/$objectKey"; // other/basic-text-object
$altObject = "alt/$objectKey"; // alt/basic-text-object
$otherAltObject = "other/alt/$objectKey"; // other/alt/basic-text-object
随后分别调用 listObjects() 枚举两个桶(底层映射为 ListObjectsV2,见 S3Service.php)。程序实际输出的对比结果如下:
Directory bucket content
other/basic-text-object
other/alt/basic-text-object
basic-text-object
alt/basic-text-object
Normal bucket content
alt/basic-text-object
basic-text-object
other/alt/basic-text-object
other/basic-text-object
差异的原因正是 "Directory" 这个名字本身:
- 普通 Bucket 以"扁平的键值对"存储对象,
ListObjects把整个 Key 当作对象标识,按**字典序(lexicographical order)**排列; - Directory Bucket 真正创建了目录/文件夹结构,Key 被当作通往对象的"路径"而非完整标识,因此枚举结果体现的是目录遍历顺序,不遵循字典序;
- 目录结构带来的好处是更快的索引、遍历与检索;桶的目录层级越深、对象越多,这种结构优势越明显。
规范文档将 "Directory!" 称为理解本步骤的钥匙,并提醒:这种结构差异会让 ListObjects 的行为与常规预期不同,可能产生"意外结果",需要开发者留意。此步骤要求捕获 NoSuchBucket 异常。
第 8 步:资源清理
程序提示用户是否清理,回答 Y/y 则调用 cleanUp()。清理顺序(对应 S3ExpressBasics.php 的 cleanUp() 方法)严格遵循"先删依赖、后删主体"的层级:
- 若创建过 VPC:先
deleteVpcEndpoint(删除 VPC Endpoint),再deleteVpc(删除 VPC); deleteStack删除 CloudFormation 堆栈(连带 IAM 用户与策略);- 依次
deleteObject删除两个桶中的 4 个对象(basic-text-object、other/...、alt/...、other/alt/...); deleteBucket删除 Directory Bucket 与普通 Bucket。
程序用 $resources 数组跟踪所有已创建资源,每个资源删除后立即 unset;清理结束时若仍有残留 Key,会输出:
There was a problem deleting some resources. The following keys are still set:
从而让用户明确知道哪些资源未能删除。规范要求此步骤处理所有与资源删除相关的异常。
集成测试 php/example_code/s3/express/tests/S3ExpressBasicsTest.php 也验证了这套流程:它预置了整段用户输入序列(含 y 应答 VPC 与清理确认),调用 runExample() 断言"无异常运行完成",并在 finally 中执行 cleanUp(),保证了演示可自动化回归。
完整控制台输出参考
规范文档给出了该场景在控制台中的完整输出样例(以 PHP 实现为例),下文完整保留:
--------------------------------------
Welcome to the Amazon S3 Express Basics demo using PHP!
--------------------------------------
Let's get started! First, please note that S3 Express One Zone works best when working within the AWS infrastructure,
specifically when working in the same Availability Zone. To see the best results in this example, and when you implement
Directory buckets into your infrastructure, it is best to put your Compute resources in the same AZ as your Directory
bucket.
1. First, we'll set up a new VPC and VPC Endpoint if this program is running in an EC2 instance in the same AZ as your Directory buckets will be.
Skipping the VPC setup. Don't forget to use this in production!
2. Policies, users, and roles with CDK.
Now, we'll set up some policies, roles, and a user. This user will only have permissions to do S3 Express One Zone actions.
3. Create an additional client using the credentials with S3 Express permissions.
This client is created with the credentials associated with the user account with the S3 Express policy attached, so it can perform S3 Express operations.
All the roles and policies were created an attached to the user. Then, a new S3 Client and Service were created using that user's credentials.
We can now use this client to make calls to S3 Express operations. Keeping permissions in mind (and adhering to least-privilege) is crucial to S3 Express.
3. Create two buckets.
Now we will create a Directory bucket, which is the linchpin of the S3 Express One Zone service.
Directory buckets behave in different ways from regular S3 buckets, which we will explore here.
We'll also create a normal bucket, put an object into the normal bucket, and copy it over to the Directory bucket.
Now, let's create the actual Directory bucket, as well as a regular bucket.
Great! Both buckets were created.
5. Create an object and copy it over.
We'll create a basic object consisting of some text and upload it to the normal bucket.
Next, we'll copy the object into the Directory bucket using the regular client.
This works fine, because Copy operations are not restricted for Directory buckets.
It worked! It's important to remember the user permissions when interacting with Directory buckets.
Instead of validating permissions on every call as normal buckets do, Directory buckets utilize the user credentials and session token to validate.
This allows for much faster connection speeds on every call. For single calls, this is low, but for many concurrent calls, this adds up to a lot of time saved.
6. Demonstrate performance difference.
Now, let's do a performance test. We'll download the same object from each bucket 1000 times and compare the total time needed. Note: the performance difference will be much more pronounced if this example is run in an EC2 instance in the same AZ as the bucket.
The directory bucket took 2464126625 nanoseconds, while the normal bucket took 1989507125.
That's a difference of -474619500 nanoseconds, or -0.4746195 seconds.
7. Populate the buckets to show the lexicographical difference.
Now let's explore how Directory buckets store objects in a different manner to regular buckets.
The key is in the name "Directory!"
Where regular buckets store their key/value pairs in a flat manner, Directory buckets use actual directories/folders.
This allows for more rapid indexing, traversing, and therefore retrieval times!
The more segmented your bucket is, with lots of directories, sub-directories, and objects, the more efficient it becomes.
This structural difference also causes ListObjects to behave differently, which can cause unexpected results.
Let's add a few more objects with layered directories as see how the output of ListObjects changes.
Directory bucket content
other/basic-text-object
other/alt/basic-text-object
basic-text-object
alt/basic-text-object
Normal bucket content
alt/basic-text-object
basic-text-object
other/alt/basic-text-object
other/basic-text-object
Notice how the normal bucket lists objects in lexicographical order, while the directory bucket does not. This is because the normal bucket considers the whole "key" to be the object identifies, while the directory bucket actually creates directories and uses the object "key" as a path to the object.
That's it for our tour of the basic operations for S3 Express One Zone.
场景涉及的 API 与元数据映射(SOS Tags)
规范文档用下表标注了本场景每个 action 对应的元数据文件与元数据键,便于自动化测试工具(SOS,Spotlight on SDKs)按操作维度校验覆盖率。原表完整继承如下:
| action | metadata file | metadata key |
|---|---|---|
CreateVpc |
ec2_metadata.yml | ec2_CreateVpc |
DescribeRouteTables |
ec2_metadata.yml | ec2_DescribeRouteTables |
CreateVpcEndpoint |
ec2_metadata.yml | ec2_CreateVpcEndpoint |
CreateBucket |
s3_metadata.yml | s3_CreateBucket |
CopyObject |
s3_metadata.yml | s3_CopyObject |
GetObject |
s3_metadata.yml | s3_GetObject |
PutObject |
s3_metadata.yml | s3_PutObject |
ListObjects |
s3_metadata.yml | s3_ListObjectsV2 |
DeleteObject |
s3_metadata.yml | s3_DeleteObject |
DeleteBucket |
s3_metadata.yml | s3_DeleteBucket |
DeleteVpcEndpoint |
ec2_metadata.yml | ec2_DeleteVpcEndpoint |
DeleteVpc |
ec2_metadata.yml | ec2_DeleteVpc |
可以看出:S3 侧覆盖了 Bucket 生命周期与对象生命周期的全部基础操作;EC2 侧覆盖了 VPC Endpoint 的创建与删除,形成"建网 → 建桶 → 读写 → 复制 → 清理"的完整闭环。
各步骤异常处理速查
规范文档为每个步骤明确了异常处理边界,整理如下便于实现时对照:
| 步骤 | 操作 | 需要捕获的异常 |
|---|---|---|
| 1 | VPC / VPC Endpoint | 无特有错误 |
| 2 | 创建策略、角色、用户、凭证 | InvalidArgumentException |
| 3 | 创建两个 S3 Client | InvalidArgumentException |
| 4 | 创建两个 Bucket | BucketAlreadyExists |
| 5 | 创建对象并复制 | 无相关特有错误 |
| 6 | 性能对比下载 | NoSuchKey |
| 7 | 填充对象并 ListObjects | NoSuchBucket |
| 8 | 清理资源 | 与资源删除相关的所有错误 |
此外,S3Service.php 的 createSession() 还会针对 NoSuchBucket 给出中文友好的提示信息("The specified bucket does not exist."),说明底层封装对会话类错误的防御性处理。
总结与延伸阅读
本文完整拆解了 S3 Express One Zone 基础场景从网络搭建到资源清理的全部 8 个步骤,并对照仓库源码确认了关键实现事实:Directory Bucket 的命名与配置结构、s3express:CreateSession 会话鉴权机制、CloudFormation 最小权限模板、两类 Bucket 在 ListObjects 排序与鉴权模型上的本质差异。
如果你想继续深入:
- 查看场景运行入口与主程序:ExpressRunner.php 与 S3ExpressBasics.php;
- 查看底层服务封装(
createBucket/createSession/listObjects等实现):php/example_code/s3/S3Service.php; - 查看最小权限 IAM 模板:resources/cfn/s3_express_basics/s3_express_template.yml;
- 查看该示例的独立说明文档:php/example_code/s3/express/README.md;
- 查看场景级概要文档:scenarios/basics/s3express/README.md。
需要特别留意的实践结论:S3 Express One Zone 的性能优势依赖"计算资源与 Directory Bucket 同 AZ"这一前提;Directory Bucket 的会话式鉴权与目录式存储,决定了权限设计和 ListObjects 用法都应与普通 Bucket 区别对待。