/ devil is in the details

silent errors & PHP stream wrappers

Did you know that the AWS PHP library for S3 includes a PHP stream wrapper? By hooking into PHP's stream extension, it lets you do things like file_get_contents('s3://mybucket/everything_is_awesome.mp3');

Any operation you can do to files using the standard file functions you can now do with files on S3. Retrofitting some legacy code? Yeah, this just saved you a bunch of work.

However there's one small thing you might not expect: some, but not all filesystem calls will silently return FALSE without so much as a whisper in the logs. If you're trying to figure out if your AWS access keys are working, this can be very, very frustrating.

The key is in the flags parameter to streamWrapper::url_stat. At a very low level, PHP developers decided that it's OK if certain stat(2)-related functions failed silently. PHP indicates this to your stream wrapper by passing the STREAM_URL_STAT_QUIET flag to url_stat.

But which functions does this happen on?

Well, this takes digging into the PHP source code. In ext/standard/filestat.c there's a macro called [IS_EXISTS_CHECK] (https://github.com/php/php-src/blob/master/ext/standard/filestat.c#L843):

#define IS_EXISTS_CHECK(__t) ((__t) == FS_EXISTS  || (__t) == FS_IS_W || (__t) == FS_IS_R || (__t) == FS_IS_X || (__t) == FS_IS_FILE || (__t) == FS_IS_DIR || (__t) == FS_IS_LINK)

Wait! Don't let your eyes glaze over, if you spend a little more time combing the source, you'll come up with these functions:

  • file_exists
  • is_writable / is_writeable
  • is_readable
  • is_executable
  • is_file
  • is_dir
  • is_link

tl;dr: use stat() not is_*()

OK, so what does all this have to do with debugging your S3 connection?

Well, this will fail silently and return FALSE:

is_file('s3://mybucket/everything_is_awesome.mp3');

This will also return FALSE but not before calling trigger_error and telling you what's going on:

stat('s3://mybucket/everything_is_awesome.mp3');

# ... in the logs:
PHP E_USER_WARNING 403 Forbidden (Request-ID: DE5C7348095B7F0C) …/vendor/aws/aws-sdk-php/src/Aws/S3/StreamWrapper.php on line 759
PHP E_WARNING stat(): stat failed for s3://mybucket/everything_is_awesome.mp3 in app.php on line 16

And now you get to debug your AWS S3 grants! sigh

Here are some links to help there:

Good luck shaving that yak!