git clone https://github.com/c4po/docker-s3fs.git export AWS_ACCESS_KEY_ID=xxxxx export AWS_SECRET_ACCESS_KEY=xxxx export SFTP_USER=admin export SFTP_PASSWORD=password export SSH_KEY=~/.ssh/id_rsa.pub export S3_BUCKET=mybucket export S3_KEY=/ make
sftp -i ~/.ssh/id_rsa -P 30022 [email protected].bb.cc.dd
s3fs-fuse allows Linux and Mac OS X to mount an S3 bucket via FUSE.
Dockerize s3fs-fuse is straightforward. However, there are some considerations about how to run s3fs and sftp server in the container. They can be both run as daemon process and foreground application. Here we choose to run s3fs as daemon process and sshd in foreground mode.
I also tried to run s3fs and sshd in 2 containers and use data volume to share data between them. However, s3fs-fuse is using libfuse to manage the filesystem mount. It cannot be recognized by docker volume.
There are 2 things we should consider when run this in Kubernetes.
We don’t want to put the AWS key to the pod definitions, so they need be read from secrets.
Load password into Kubernetes secets.
apiVersion: v1 kind: Secret metadata: name: s3fs-secret type: Opaque data: sftp_user: YWRtaW4= sftp_password: cGFzc3dvcmQ= aws_accesskey: xxxx aws_secretkey: xxxx
use secrets as environment variables:
spec: containers: env: - name: AWS_ACCESSKEY valueFrom: secretKeyRef: name: s3fs-secret key: aws_accesskey
We need give the container enough privilege to allow s3fs-fuse to manage filesystem.
spec: containers: securityContext: privileged: true
When we manually create secrets in Kubernetes, the value need be base64 encoded.
echo 'admin' | base64 is different than
echo -n 'admin' | base64. We don’t want to have the newline character in the value. So be careful to use
echo -n or
printf to remove them.
Written by Max Cai Github