{"id":5264,"date":"2017-02-12T00:01:43","date_gmt":"2017-02-11T15:01:43","guid":{"rendered":"http:\/\/www.magtranetwork.com\/?p=5264"},"modified":"2017-02-10T13:49:58","modified_gmt":"2017-02-10T04:49:58","slug":"check_s3_total_objects_size_and_number","status":"publish","type":"post","link":"https:\/\/www.magtranetwork.com\/aws\/check_s3_total_objects_size_and_number.html","title":{"rendered":"Amazon S3\u306e\u5404\u30d0\u30b1\u30c3\u30c8\u6bce\u306e\u5408\u8a08\u5bb9\u91cf\u30fb\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u6570\u3092\u96c6\u8a08\u3057\u3001\u53d6\u5f97\u3059\u308bAWS CLI\u3092\u7528\u3044\u305f\u30b7\u30a7\u30eb\u30b9\u30af\u30ea\u30d7\u30c8(\u30d0\u30c3\u30c1\u51e6\u7406\u30d7\u30ed\u30b0\u30e9\u30e0) \uff5e\u30d0\u30b1\u30c3\u30c8\u540d\u3092\u6307\u5b9a\u3057\u305f\u691c\u7d22\u3001\u30d0\u30b1\u30c3\u30c8\u540d\u304b\u3089\u30ea\u30fc\u30b8\u30e7\u30f3\u3092\u81ea\u52d5\u5224\u5b9a\u3001\u30d0\u30b1\u30c3\u30c8\u30b5\u30a4\u30ba\u30fb\u30d5\u30a1\u30a4\u30eb\u6570\u3092\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u3067\u8a08\u7b97\uff5e"},"content":{"rendered":"
AWS\u306b\u304a\u3044\u3066Amazon S3\u306f\u5358\u7d14\u306a\u30d5\u30a1\u30a4\u30eb\u4fdd\u5b58\u306e\u30b9\u30c8\u30ec\u30fc\u30b8\u3060\u3051\u3067\u306f\u306a\u304f\u3001\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u3001\u30a6\u30a7\u30d6\u30db\u30b9\u30c6\u30a3\u30f3\u30b0\u3001\u30d0\u30c3\u30c1\u51e6\u7406\u30fb\u30d3\u30c3\u30b0\u30c7\u30fc\u30bf\u306e\u4e2d\u9593\u30c7\u30fc\u30bf\u306e\u4fdd\u5b58\u306a\u3069\u69d8\u3005\u306a\u7528\u9014\u306b\u4f7f\u7528\u3055\u308c\u3001AWS\u3092\u7528\u3044\u305f\u30b7\u30b9\u30c6\u30e0\u306e\u6838\u306b\u4f4d\u7f6e\u3065\u3051\u3089\u308c\u308b\u3053\u3068\u3082\u591a\u304f\u3042\u308b\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n
\u305d\u306e\u3088\u3046\u306a\u30c7\u30fc\u30bf\u30bd\u30fc\u30b9\u306e\u4e2d\u5fc3\u3068\u3082\u306a\u308bAmazon S3\u3067\u3059\u304c\u3001\u4fdd\u5b58\u3067\u304d\u308b\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u5bb9\u91cf\u3001\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u6570\u306b\u5236\u9650\u306f\u306a\u3044\u305f\u3081\u3001\u6ce8\u610f\u3057\u3066\u3044\u306a\u3044\u3068\u30d0\u30c3\u30c1\u51e6\u7406\u7b49\u3067\u5927\u91cf\u306e\u30c7\u30fc\u30bf\u304c\u4fdd\u5b58\u3055\u308c\u3066\u3044\u308b\u72b6\u6cc1\u3082\u8003\u3048\u3089\u308c\u307e\u3059\u3002<\/p>\n
\u7279\u306bAmazon S3\u306f\u4fdd\u5b58\u30c7\u30fc\u30bf\u5bb9\u91cf\u306b\u5bfe\u3059\u308b\u5f93\u91cf\u8ab2\u91d1\u5236\u306a\u306e\u3067\u4fdd\u5b58\u3059\u308b\u30c7\u30fc\u30bf\u5bb9\u91cf\u304c\u591a\u3051\u308c\u3070\u591a\u3044\u307b\u3069\u8cbb\u7528\u304c\u304b\u304b\u308a\u307e\u3059\u3002<\/p>\n
Amazon S3\u306b\u4fdd\u5b58\u3055\u308c\u3066\u3044\u308b\u30d0\u30b1\u30c3\u30c8\u6bce\u306e\u306e\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u5bb9\u91cf\u3001\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u6570\u3092\u77e5\u308b\u65b9\u6cd5\u306f\u5927\u304d\u304f\u5206\u3051\u3066\u4e0b\u8a18\u306e2\u3064\u306e\u65b9\u6cd5\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n
\u305f\u3060\u3001CloudWatch\u306e\u30e1\u30c8\u30ea\u30af\u30b9\u306f\u81ea\u52d5\u7684\u306b\u96c6\u8a08\u3057\u3066\u304f\u308c\u308b\u3068\u3044\u3046\u30e1\u30ea\u30c3\u30c8\u306e\u4e00\u65b9\u30671\u65e5\u306b1\u56de\u306e\u96c6\u8a08\u7d50\u679c\u3057\u304b\u8868\u793a\u3055\u308c\u306a\u3044\u3068\u3044\u3046\u30c7\u30e1\u30ea\u30c3\u30c8\u304c\u3042\u308b\u305f\u3081\u3001\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u3067\u5bb9\u91cf\u3084\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u6570\u3092\u77e5\u308a\u305f\u3044\u5834\u5408\u306f\u4f7f\u7528\u3067\u304d\u307e\u305b\u3093\u3002<\/p>\n
\u4eca\u56de\u306f\u30b7\u30b9\u30c6\u30e0\u3084\u30d0\u30c3\u30c1\u51e6\u7406\u7b49\u3067\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u306bS3\u30d0\u30b1\u30c3\u30c8\u6bce\u306e\u5bb9\u91cf\u30fb\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u6570\u304c\u77e5\u308a\u305f\u3044\u5834\u5408\u306b\u4f7f\u3048\u308bAWS CLI\u3067S3\u306e\u30d0\u30b1\u30c3\u30c8\u914d\u4e0b\u306e\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u306b\u5bfe\u3057\u3066\u518d\u5e30\u7684\u306b\u5bb9\u91cf\u3068\u6570\u3092\u96c6\u8a08\u3059\u308b\u30b7\u30a7\u30eb\u30b9\u30af\u30ea\u30d7\u30c8(\u30d0\u30c3\u30c1\u51e6\u7406\u30d7\u30ed\u30b0\u30e9\u30e0)\u3092\u5099\u5fd8\u9332\u3068\u3057\u3066\u8a18\u8f09\u3057\u307e\u3059\u3002<\/p>\n
\u57fa\u672c\u65b9\u91dd\u3068\u3057\u3066\u306f\u30d0\u30b1\u30c3\u30c8\u540d\u4e00\u89a7\u3092\u5168\u3066\u53d6\u5f97\u3057\u3001\u691c\u7d22\u30ad\u30fc\u30ef\u30fc\u30c9\u304c\u5f15\u6570\u306b\u3042\u308c\u3070\u3001\u305d\u308c\u3092\u6761\u4ef6\u306b\u30d0\u30b1\u30c3\u30c8\u3092\u7d5e\u8fbc\u307f\u3001\u5bfe\u8c61\u306e\u30d0\u30b1\u30c3\u30c8\u306b\u5bfe\u3057\u3066AWS CLI\u3067\u30d0\u30b1\u30c3\u30c8\u914d\u4e0b\u306e\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u5bb9\u91cf\u30fb\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u6570\u3092\u518d\u5e30\u7684\u306b\u96c6\u8a08\u3059\u308b\u3068\u3044\u3046\u6d41\u308c\u3067\u3059\u3002<\/p>\n
\u51e6\u7406\u306e\u5185\u5bb9\u306f\u57fa\u672c\u65b9\u91dd\u3068\u540c\u3058\u3067\u3059\u304c\u3001AWS CLI\u3067\u30d0\u30b1\u30c3\u30c8\u306e\u60c5\u5831\u3092\u53d6\u5f97\u3059\u308b\u5834\u5408\u306bEU\u7cfb\u306e\u30ea\u30fc\u30b8\u30e7\u30f3\u3067\u306f\u30ea\u30fc\u30b8\u30e7\u30f3\u6307\u5b9a\u3092\u3059\u308b\u6587\u5b57\u5217\u304c\u7121\u3044\u3068\u30a8\u30e9\u30fc\u304c\u767a\u751f\u3059\u308b\u5834\u5408\u304c\u3042\u308b\u305f\u3081\u3001\u5404S3\u30d0\u30b1\u30c3\u30c8\u540d\u304b\u3089\u30ea\u30fc\u30b8\u30e7\u30f3\u6307\u5b9a\u3092\u3059\u308b\u6587\u5b57\u5217\u3092\u751f\u6210\u3059\u308b\u95a2\u6570\u3092\u4f5c\u6210\u3057\u3066AWS CLI\u5b9f\u884c\u6642\u306b\u4ed8\u52a0\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n
\r\n[magtranetwork@localhost ~]# vim check_s3_total_objects_size.sh\r\n<\/pre>\n\r\n#!\/bin\/bash\r\n\r\n# S3\u30d0\u30b1\u30c3\u30c8\u540d\u304b\u3089\u30ea\u30fc\u30b8\u30e7\u30f3\u30aa\u30d7\u30b7\u30e7\u30f3\u6587\u5b57\u5217\u3092\u4f5c\u6210\u3059\u308b\u95a2\u6570\r\n# \\`bucket_region ${BUCKET}\\`\r\nbucket_region () {\r\n REGION=`aws s3api get-bucket-location --output text --bucket ${1}`\r\n if [ "${REGION}" = "None" -o "${REGION}" = "" ]; then\r\n REGION_STR=""\r\n else\r\n REGION_STR="--region ${REGION}"\r\n fi\r\n echo ${REGION_STR}\r\n}\r\n\r\n#\u691c\u7d22\u6587\u5b57\u5217\u3092\u5f15\u6570\u304b\u3089\u53d6\u5f97\u3059\u308b\u3002\r\nPREFIX=$1\r\n\r\nif [ "${PREFIX}" = "" ]; then\r\n #\u691c\u7d22\u6587\u5b57\u5217\u304c\u6307\u5b9a\u3055\u308c\u3066\u3044\u306a\u3044\u5834\u5408\u306f\u5168\u3066\u306e\u30d0\u30b1\u30c3\u30c8\u540d\u3092\u53d6\u5f97\u3059\u308b\u3002\r\n BUCKETS=(`aws s3 ls s3:\/\/ | awk '{print $NF}' | sed -e 's\/ \/\/g'`)\r\nelse\r\n #\u691c\u7d22\u6587\u5b57\u5217\u304c\u6307\u5b9a\u3055\u308c\u3066\u3044\u308b\u5834\u5408\u306f\u691c\u7d22\u6587\u5b57\u5217\u3067\u30d0\u30b1\u30c3\u30c8\u540d\u3092\u691c\u7d22\u3057\u3001\u8a72\u5f53\u3059\u308b\u3082\u306e\u3092\u53d6\u5f97\u3059\u308b\u3002\r\n BUCKETS=(`aws s3 ls s3:\/\/ | awk '{print $NF}' | sed -e 's\/ \/\/g' | grep "^${PREFIX}$"`)\r\nfi\r\n\r\n#\u53d6\u5f97\u3057\u305f\u30d0\u30b1\u30c3\u30c8\u540d\u914d\u5217\u304c\u7a7a\u306e\u5834\u5408\u306f\u51e6\u7406\u3092\u7d42\u4e86\u3002\r\nif [ "${BUCKETS}" = "" ]; then\r\n echo "Target Bucket is not exists in S3."\r\n exit 1\r\nfi\r\n\r\n#\u53d6\u5f97\u3057\u305f\u30d0\u30b1\u30c3\u30c8\u540d\u914d\u5217\u304b\u3089\u30d0\u30b1\u30c3\u30c8\u540d\u3092\u7e70\u308a\u8fd4\u3057\u53d6\u5f97\u3057\u51e6\u7406\u3092\u3059\u308b\u3002\r\nfor BUCKET in ${BUCKETS[@]}; do\r\n #\u30d0\u30b1\u30c3\u30c8\u914d\u4e0b\u306e\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u306e\u5bb9\u91cf\u30fb\u6570\u3092\u518d\u5e30\u7684(--recursive)\u306b\u96c6\u8a08(--summarize)\u3057\u3001\u4eba\u9593\u304c\u8aad\u3081\u308b\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u3067\u8868\u793a(--human-readable)\u3059\u308b\u3002\r\n INFO=`aws s3 \\`bucket_region ${BUCKET}\\` ls --recursive --human-readable --summarize s3:\/\/${BUCKET} | grep "Total" | sed -e 's\/ \/\/g' | tr '\\n' ' '`\r\n echo "BucketName:${BUCKET} ${INFO}"\r\ndone\r\n\r\n<\/pre>\nAmazon S3\u306e\u5404\u30d0\u30b1\u30c3\u30c8\u6bce\u306e\u5408\u8a08\u5bb9\u91cf\u30fb\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u6570\u3092\u96c6\u8a08\u3057\u3001\u53d6\u5f97\u3059\u308bAWS CLI\u3092\u7528\u3044\u305f\u30b7\u30a7\u30eb\u30b9\u30af\u30ea\u30d7\u30c8(\u30d0\u30c3\u30c1\u51e6\u7406\u30d7\u30ed\u30b0\u30e9\u30e0)\u306e\u5b9f\u884c\u4f8b<\/h3>\n
\u4e0a\u8a18\u306e\u30b7\u30a7\u30eb\u30b9\u30af\u30ea\u30d7\u30c8(\u30d0\u30c3\u30c1\u51e6\u7406\u30d7\u30ed\u30b0\u30e9\u30e0)\u306e\u5b9f\u884c\u306f\u5f15\u6570\u3092\u6307\u5b9a\u3057\u306a\u3051\u308c\u3070AWS\u30a2\u30ab\u30a6\u30f3\u30c8\u306b\u5b58\u5728\u3059\u308b\u5168\u3066\u306eS3\u30d0\u30b1\u30c3\u30c8\u306b\u5bfe\u3057\u3066\u96c6\u8a08\u3092\u884c\u3044\u307e\u3059\u3002<\/p>\n
S3\u30d0\u30b1\u30c3\u30c8\u3092\u7d5e\u308a\u8fbc\u3080\u5834\u5408\u306f\u5f15\u6570\u306b\u691c\u7d22\u6587\u5b57\u5217\u3092\u6307\u5b9a\u3057\u3066\u5b9f\u884c\u3057\u307e\u3059\u3002<\/p>\n
\uff1c\u5168\u4ef6\u8868\u793a\u3059\u308b\u5834\u5408\u306e\u4f8b\uff1e<\/p>\n
\r\n[magtranetwork@localhost ~]# chmod 755 check_s3_total_objects_size.sh\r\n[magtranetwork@localhost ~]# .\/check_s3_total_objects_size.sh\r\nBucketName:your-bucket-name-01 TotalObjects:12161 TotalSize:3.9GiB\r\nBucketName:your-bucket-name-02 TotalObjects:12953 TotalSize:3.9GiB\r\nBucketName:your-bucket-name-03 TotalObjects:47 TotalSize:141.9KiB\r\nBucketName:your-bucket-name-04 TotalObjects:42 TotalSize:2.3GiB\r\nBucketName:your-bucket-name-05 TotalObjects:57 TotalSize:44.6GiB\r\nBucketName:your-bucket-name-06 TotalObjects:162 TotalSize:60.2MiB\r\nBucketName:your-bucket-name-07 TotalObjects:4 TotalSize:77.9KiB\r\nBucketName:your-bucket-name-08 TotalObjects:66537 TotalSize:23.2GiB\r\nBucketName:your-bucket-name-09 TotalObjects:2 TotalSize:20.4KiB\r\nBucketName:your-bucket-name-10 TotalObjects:4335 TotalSize:215.9MiB\r\n<\/pre>\n\uff1c\u7279\u5b9a\u306e\u6587\u5b57\u5217\u3067\u30d0\u30b1\u30c3\u30c8\u3092\u691c\u7d22\u3059\u308b\u5834\u5408\u306e\u4f8b\uff1e<\/p>\n
\r\n[magtranetwork@localhost ~]# chmod 755 check_s3_total_objects_size.sh\r\n[magtranetwork@localhost ~]# .\/check_s3_total_objects_size.sh your-bucket-name-05\r\nBucketName:your-bucket-name-05 TotalObjects:57 TotalSize:44.6GiB\r\n<\/pre>\n\u5fdc\u7528\u306e\u53ef\u80fd\u6027<\/h3>\n
\u4eca\u56de\u306fS3\u30d0\u30b1\u30c3\u30c8\u306e\u76f4\u4e0b\u3092\u518d\u5e30\u7684\u306b\u96c6\u8a08\u3059\u308b\u3082\u306e\u3092\u4f5c\u6210\u3057\u307e\u3057\u305f\u304c\u3001\u5b9f\u969b\u306e\u96c6\u8a08\u3092\u884c\u3063\u3066\u3044\u308bAWS CLI\u30b3\u30de\u30f3\u30c9\u3067\u300cs3:\/\/${BUCKET}\/dir\/\u300d\u306e\u3088\u3046\u306b\u30b5\u30d6\u30d5\u30a9\u30eb\u30c0\u3092\u6307\u5b9a\u3059\u308b\u3068\u305d\u306e\u914d\u4e0b\u3092\u96c6\u8a08\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/p>\n
\u8907\u6570\u30e6\u30fc\u30b6\u30fc\u304cS3\u30d0\u30b1\u30c3\u30c8\u3092\u4f7f\u7528\u3057\u3066\u3044\u305f\u308a\u3001\u30d0\u30c3\u30c1\u304cS3\u30d0\u30b1\u30c3\u30c8\u914d\u4e0b\u306b\u81ea\u52d5\u7684\u306b\u4e2d\u9593\u30d5\u30a1\u30a4\u30eb\u3092\u4f5c\u6210\u3059\u308b\u5834\u5408\u306a\u3069\u5bb9\u91cf\u76e3\u8996\u3084\u901a\u77e5\u3092\u884c\u3044\u305f\u3044\u5834\u5408\u306b\u4eca\u56de\u306e\u65b9\u6cd5\u306f\u6d3b\u7528\u3067\u304d\u308b\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"
AWS\u306b\u304a\u3044\u3066Amazon S3\u306f\u5358\u7d14\u306a\u30d5\u30a1\u30a4\u30eb\u4fdd\u5b58\u306e\u30b9\u30c8\u30ec\u30fc\u30b8\u3060\u3051\u3067\u306f\u306a\u304f\u3001\u30d0\u30c3\u30af\u30a2\u30c3\u30d7\u3001\u30a6\u30a7\u30d6\u30db\u30b9\u30c6\u30a3\u30f3\u30b0\u3001\u30d0\u30c3\u30c1\u51e6\u7406\u30fb\u30d3\u30c3\u30b0\u30c7\u30fc\u30bf\u306e\u4e2d\u9593\u30c7\u30fc\u30bf\u306e\u4fdd\u5b58\u306a\u3069\u69d8\u3005\u306a\u7528\u9014\u306b\u4f7f\u7528\u3055\u308c\u3001AWS\u3092\u7528\u3044\u305f\u30b7\u30b9\u30c6\u30e0\u306e\u6838\u306b\u4f4d\u7f6e\u3065\u3051\u3089\u308c […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":""},"categories":[3,81,22,79,38,82],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/posts\/5264"}],"collection":[{"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/comments?post=5264"}],"version-history":[{"count":4,"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/posts\/5264\/revisions"}],"predecessor-version":[{"id":5270,"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/posts\/5264\/revisions\/5270"}],"wp:attachment":[{"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/media?parent=5264"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/categories?post=5264"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.magtranetwork.com\/wp-json\/wp\/v2\/tags?post=5264"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}