I am trying to mount a google bucket using gcsfuse. I am able to mount it but am getting an input/output error message when I try to use touch to test making files inside of it.
The command below shows that there are no errors trying to connect to the google cloud at all
my_name@deployment-test:~$ sudo gcsfuse -o rw -o allow_other -file-mode=777 -dir-mode=777 --foreground --debug_gcs --debug_http --debug_fuse --debug_invariants company-name-deployment-test test2
The Command gives the results
Using mount point: /home/my_name/test2
Opening GCS connection...
WARNING: gcsfuse invoked as root. This will cause all files to be owned by
root. If this is not what you intended, invoke gcsfuse as the user that will
be interacting with the file system.
Opening bucket...
gcs: Req 0x0: <- ListObjects()
http: ========== REQUEST:
GET http://www.googleapis.com/storage/v1/b/company-name-deployment-test/o?maxResults=1&projection=full HTTP/1.1
Host: www.googleapis.com
User-Agent: gcsfuse/0.0
Authorization: Bearer ya29.EmQkBGU8LBXoLOWE6bt7QU7Ulg9k6_ALV9shBuGILqzZxEf4mE0xXIVuelOscxJiXwLpvNAPO23oNT3oqE1YT8KJxmvcGlhv1XwAV3jOqO2-2NMeB9w1pLreGGXNLMBcxG7NoUIX
Accept-Encoding: gzip
http: ========== RESPONSE:
HTTP/2.0 200 OK
Content-Length: 31
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Wed, 05 Apr 2017 14:53:47 GMT
Expires: Wed, 05 Apr 2017 14:53:47 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: AEnB2Uo_mWOeNeVYEKUDSUgXG5l6OW3rdBghHeFlfzUjDMGMSl4934ZLq57ujokZuJoSAefKXIlqZTjURPcx_HTq0obPy94PgQ
{
"kind": "storage#objects"
}
http: ====================
gcs: Req 0x0: -> ListObjects() (80.50838ms): OK
Mounting file system...
fuse_debug: Op 0x00000001 connection.go:395] <- init
fuse_debug: Op 0x00000001 connection.go:474] -> OK ()
File system has been successfully mounted.
trying to touch gives:
touch test2/test.txt
touch: cannot touch ‘test2/test.txt’: Input/output error
Since my credentials appear to be okay from the debug output What else could be causing this Input/output error message?
After reading Jacobsa's comment I realized I was not getting all the information I could I opened a second ssh connection once the system was mounted to run my command while this was still up and trying to touch the file yielded:
fuse_debug: Op 0x00000003 connection.go:395] <- LookUpInode (parent 1, name "test.txt")
gcs: Req 0x1: <- StatObject("test.txt")
gcs: Req 0x2: <- StatObject("test.txt/")
http: ========== REQUEST:
GET http://www.googleapis.com/storage/v1/b/tembo-deployment-test/o/test.txt?projection=full HTTP/1.1
Host: www.googleapis.com
User-Agent: gcsfuse/0.0
Authorization: Bearer ya29.EmQmBNtnKIPWQvvhLiCU-178OTVIA44gXrXRzQaNb5tBTfRpusKgmgmF83FR6QCI1Cg6jACSkZ3iCWyqarhS7WSuPfMUih0gZpKXGSdSg4fQ6VbFQnj7ru24k4erRUTVdR04NrcH
Accept-Encoding: gzip
http: ========== REQUEST:
GET http://www.googleapis.com/storage/v1/b/tembo-deployment-test/o/test.txt%2F?projection=full HTTP/1.1
Host: www.googleapis.com
User-Agent: gcsfuse/0.0
Authorization: Bearer ya29.EmQmBNtnKIPWQvvhLiCU-178OTVIA44gXrXRzQaNb5tBTfRpusKgmgmF83FR6QCI1Cg6jACSkZ3iCWyqarhS7WSuPfMUih0gZpKXGSdSg4fQ6VbFQnj7ru24k4erRUTVdR04NrcH
Accept-Encoding: gzip
http: ========== RESPONSE:
HTTP/2.0 404 Not Found
Content-Length: 165
Cache-Control: private, max-age=0
Content-Type: application/json; charset=UTF-8
Date: Fri, 07 Apr 2017 15:29:07 GMT
Expires: Fri, 07 Apr 2017 15:29:07 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: AEnB2Uo2tVrtHfLuSn2RDpYqO3bgg6gEPL1fjOO2wdBgKGGyNzKHVQXlEoh4ObOx0YH1soFOQijWzXuewbAtMBdNcSxmeHzuGg
{
"error": {
"errors": [
{
"domain": "global",
"reason": "notFound",
"message": "Not Found"
}
],
"code": 404,
"message": "Not Found"
}
}
http: ====================
gcs: Req 0x2: -> StatObject("test.txt/") (54.298126ms): gcs.NotFoundError: googleapi: Error 404: Not Found, notFound
http: ========== RESPONSE:
HTTP/2.0 404 Not Found
Content-Length: 165
Cache-Control: private, max-age=0
Content-Type: application/json; charset=UTF-8
Date: Fri, 07 Apr 2017 15:29:07 GMT
Expires: Fri, 07 Apr 2017 15:29:07 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: AEnB2UrAcoLYqm97VH1sDql_CLiVM3_q0EXW_gy1VaxmWJOLksZkcIV78WRFN-Y9hkWmDKN1LJfSz767jkWAhomwoaLqwlFeyw
{
"error": {
"errors": [
{
"domain": "global",
"reason": "notFound",
"message": "Not Found"
}
],
"code": 404,
"message": "Not Found"
}
}
http: ====================
gcs: Req 0x1: -> StatObject("test.txt") (57.69261ms): gcs.NotFoundError: googleapi: Error 404: Not Found, notFound
fuse_debug: Op 0x00000003 connection.go:476] -> Error: "no such file or directory"
fuse_debug: Op 0x00000004 connection.go:395] <- CreateFile (parent 1, name "test.txt")
gcs: Req 0x3: <- CreateObject("test.txt")
http: ========== REQUEST:
POST http://www.googleapis.com/upload/storage/v1/b/tembo-deployment-test/o?ifGenerationMatch=0&projection=full&uploadType=resumable HTTP/1.1
Host: www.googleapis.com
User-Agent: gcsfuse/0.0
Content-Length: 156
Authorization: Bearer ya29.EmQmBNtnKIPWQvvhLiCU-178OTVIA44gXrXRzQaNb5tBTfRpusKgmgmF83FR6QCI1Cg6jACSkZ3iCWyqarhS7WSuPfMUih0gZpKXGSdSg4fQ6VbFQnj7ru24k4erRUTVdR04NrcH
Content-Type: application/json
X-Upload-Content-Type: text/plain; charset=utf-8
Accept-Encoding: gzip
{"bucket":"tembo-deployment-test","contentType":"text/plain; charset=utf-8","metadata":{"gcsfuse_mtime":"2017-04-07T15:29:07.544167408Z"},"name":"test.txt"}
http: ========== RESPONSE:
HTTP/2.0 403 Forbidden
Content-Length: 208
Content-Type: application/json; charset=UTF-8
Date: Fri, 07 Apr 2017 15:29:07 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
Www-Authenticate: Bearer realm="https://accounts.google.com/", error=insufficient_scope, scope="https://www.googleapis.com/auth/cloud-platform"
X-Guploader-Uploadid: AEnB2UpoWv_gpFyC_fxo8wV5bFBRUX9DLTWEzmNcymxDjERaaml3m-hv4BBmt_kErbFCYj8SYEwka0rkvomWyz1DbusJr-giLw
{
"error": {
"errors": [
{
"domain": "global",
"reason": "insufficientPermissions",
"message": "Insufficient Permission"
}
],
"code": 403,
"message": "Insufficient Permission"
}
}
http: ====================
gcs: Req 0x3: -> CreateObject("test.txt") (14.054889ms): googleapi: Error 403: Insufficient Permission, insufficientPermissions
fuse_debug: Op 0x00000004 connection.go:476] -> Error: "CreateChildFile: googleapi: Error 403: Insufficient Permission, insufficientPermissions"
fuse: 2017/04/07 15:29:07.567837 *fuseops.CreateFileOp error: CreateChildFile: googleapi: Error 403: Insufficient Permission, insufficientPermissions
fuse_debug: Op 0x00000005 connection.go:395] <- LookUpInode (parent 1, name "test.txt")
gcs: Req 0x4: <- StatObject("test.txt")
http: ========== REQUEST:
GET http://www.googleapis.com/storage/v1/b/tembo-deployment-test/o/test.txt?projection=full HTTP/1.1
Host: www.googleapis.com
User-Agent: gcsfuse/0.0
Authorization: Bearer ya29.EmQmBNtnKIPWQvvhLiCU-178OTVIA44gXrXRzQaNb5tBTfRpusKgmgmF83FR6QCI1Cg6jACSkZ3iCWyqarhS7WSuPfMUih0gZpKXGSdSg4fQ6VbFQnj7ru24k4erRUTVdR04NrcH
Accept-Encoding: gzip
http: ========== RESPONSE:
HTTP/2.0 404 Not Found
Content-Length: 165
Cache-Control: private, max-age=0
Content-Type: application/json; charset=UTF-8
Date: Fri, 07 Apr 2017 15:29:07 GMT
Expires: Fri, 07 Apr 2017 15:29:07 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: AEnB2UqxK0wYlDTOf4vT2j_X40E57dJJqEZf8YQL67c_x_3bURpf0xJVJIJuPpc17k0jHUiOAbk4JasDqZd9uD-NK0XNFODJXQ
{
"error": {
"errors": [
{
"domain": "global",
"reason": "notFound",
"message": "Not Found"
}
],
"code": 404,
"message": "Not Found"
}
}
http: ====================
gcs: Req 0x4: -> StatObject("test.txt") (98.990226ms): gcs.NotFoundError: googleapi: Error 404: Not Found, notFound
fuse_debug: Op 0x00000005 connection.go:476] -> Error: "no such file or directory"
So I need to grant permissions to the account I'm connecting with.
After running $ gcsfuse nmjcloud_jar_test /gcp/gcsfuse/mount --key-file="/home/niaomingjian/gcpconfig/My First Project-6f9cff47c4f0.json"
,
$ touch /gcp/gcsfuse/mount/test.txt
created the test.txt file successfully on my test environment.
I've tested it and it is a matter of credentials. I've come to the conclusion with two attempts.
The first attempt:
Result: Only ls didn't fail, and the rest gave the I/O error.
Then, in the second attempt I tried:
I restarted the instance (to unmount the bucket)
I downloaded a service account key with all the permissions active within the compute instance, and loaded it as a GOOGLE APPLICATION CREDENTIALS, the code was:
$export GOOGLE_APPLICATION_CREDENTIALS=.json
*Please notice I set "complete path", meaning using a relative path like "./key.json" is insufficient. The real complete path must be configured or it won't work. It must be absolute like: "home/ggrimaldo/key.json".
Conclusion: Even though gcsfuse does throw an I/O problem, it is instead credentials problem.
to create the json file you
gcloud iam service-accounts keys create ~/key.json --iam-account 245583959095-com
if you don't know what account you are use
gcloud auth list
User contributions licensed under CC BY-SA 3.0