How to get a GCP Bearer token programmatically with python
gcloud auth print-access-token
gives me a Bearer token that I can use later on; however, this is a shell command. How would I obtain one programmatically via the Google Cloud Python API?
I see a prior example using oauth2client, but oauth2client
is now deprecated. How would I do this with google.auth and oauthlib?
python google-cloud-platform google-iam
add a comment |
gcloud auth print-access-token
gives me a Bearer token that I can use later on; however, this is a shell command. How would I obtain one programmatically via the Google Cloud Python API?
I see a prior example using oauth2client, but oauth2client
is now deprecated. How would I do this with google.auth and oauthlib?
python google-cloud-platform google-iam
Do you want to get this in a GCP environment like Compute Engine, App Engine, Cloud Functions, etc? Or outside GCP, like on your local machine?
– Dustin Ingram
Nov 25 '18 at 22:10
I'm building an app/script inside a GCP Compute Engine instance with a service account. That's my first preference. I'm also interested to know how it would be like if it was run outside GCP on my local machine too.
– indraniel
Nov 25 '18 at 22:13
add a comment |
gcloud auth print-access-token
gives me a Bearer token that I can use later on; however, this is a shell command. How would I obtain one programmatically via the Google Cloud Python API?
I see a prior example using oauth2client, but oauth2client
is now deprecated. How would I do this with google.auth and oauthlib?
python google-cloud-platform google-iam
gcloud auth print-access-token
gives me a Bearer token that I can use later on; however, this is a shell command. How would I obtain one programmatically via the Google Cloud Python API?
I see a prior example using oauth2client, but oauth2client
is now deprecated. How would I do this with google.auth and oauthlib?
python google-cloud-platform google-iam
python google-cloud-platform google-iam
edited Nov 26 '18 at 1:59
Maxim
1,553211
1,553211
asked Nov 25 '18 at 22:00
indranielindraniel
726
726
Do you want to get this in a GCP environment like Compute Engine, App Engine, Cloud Functions, etc? Or outside GCP, like on your local machine?
– Dustin Ingram
Nov 25 '18 at 22:10
I'm building an app/script inside a GCP Compute Engine instance with a service account. That's my first preference. I'm also interested to know how it would be like if it was run outside GCP on my local machine too.
– indraniel
Nov 25 '18 at 22:13
add a comment |
Do you want to get this in a GCP environment like Compute Engine, App Engine, Cloud Functions, etc? Or outside GCP, like on your local machine?
– Dustin Ingram
Nov 25 '18 at 22:10
I'm building an app/script inside a GCP Compute Engine instance with a service account. That's my first preference. I'm also interested to know how it would be like if it was run outside GCP on my local machine too.
– indraniel
Nov 25 '18 at 22:13
Do you want to get this in a GCP environment like Compute Engine, App Engine, Cloud Functions, etc? Or outside GCP, like on your local machine?
– Dustin Ingram
Nov 25 '18 at 22:10
Do you want to get this in a GCP environment like Compute Engine, App Engine, Cloud Functions, etc? Or outside GCP, like on your local machine?
– Dustin Ingram
Nov 25 '18 at 22:10
I'm building an app/script inside a GCP Compute Engine instance with a service account. That's my first preference. I'm also interested to know how it would be like if it was run outside GCP on my local machine too.
– indraniel
Nov 25 '18 at 22:13
I'm building an app/script inside a GCP Compute Engine instance with a service account. That's my first preference. I'm also interested to know how it would be like if it was run outside GCP on my local machine too.
– indraniel
Nov 25 '18 at 22:13
add a comment |
1 Answer
1
active
oldest
votes
The answer depends on your environment and how you want to create / obtain credentials.
What are Google Cloud Credentials?
Google Cloud credentials are an OAuth 2.0 token. This token has at a minimum an Access Token
and optionally a Refresh Token
, Client ID Token
, and supporting parameters such as expiration
, Service Account Email
or Client Email
, etc.
The important item in Google Cloud APIs is the Access Token
. This token is what authorizes access to the cloud. This token can be used in programs such as curl
, software such as python
, etc and does not require an SDK. The Access Token
is used in the HTTP Authorization
header.
What is an Access Token?
An access token is an opaque value generated by Google that is derived from a Signed JWT, more correctly called JWS. A JWT consists of a header and claims (the payload) Json structures. These two Json structures are signed with the Service Account's Private Key. These values are base64 encoded and concatenated to create the Access Key.
The format of an Access Token is: base64(header) + '.' + base64(payload) + '.' + base64(signature)
.
Here is an example JWT:
Header:
{
"alg": "RS256",
"typ": "JWT",
"kid": "42ba1e234ac91ffca687a5b5b3d0ca2d7ce0fc0a"
}
Payload:
{
"iss": "myservice@myproject.iam.gserviceaccount.com",
"iat": 1493833746,
"aud": "myservice.appspot.com",
"exp": 1493837346,
"sub": "myservice@myproject.iam.gserviceaccount.com"
}
Using an Access Token:
Example that will start a VM instance. Replace PROJECT_ID, ZONE and INSTANCE_NAME. This example is for Windows.
curl -v -X GET -H "Authorization: Bearer <access_token_here>" ^
https://www.googleapis.com/compute/v1/projects/%PROJECT_ID%/zones/%ZONE%/instances/%INSTANCE_NAME%/start
Compute Engine Service Account:
Dustin's answer is correct for this case, but I will include for completeness with some additional information.
These credentials are automatically created for you by GCP and are obtained from the VM Instance metadata. Permissions are controlled by Cloud API access scopes
in the Google Console.
However, these credentials have some limitations. To modify the credentials you must stop the VM Instance first. Additionally, not all permissions (roles) are supported.
from google.auth import compute_engine
cred = compute_engine.Credentials()
Service Account Credentials:
Until you understand all of the types of credentials and their use cases, these are the credentials that you will use for everything except for gcloud
and gsutil
. Understanding these credentials will make working with Google Cloud much simpler when writing programs. Obtaining credentials from a Google Service Account Json file is easy. The only item to make note of is that credentials expire (typically 60 minutes) and either need to be refreshed or recreated.
gcloud auth print-access-token
is NOT recommended. Service Account Credentials are the recommended method by Google.
These credentials are created by the Console, gcloud or via programs / APIs. Permissions are assigned to the creditials by IAM and function inside Compute Engine, App Engine, Firestore, Kubernetes, etc. as well as other environments outside of Google Cloud. These credentials are downloaded from Google Cloud and stored in a Json file. Notice the scopes
parameter. This defines permissions that are granted to the resulting credentials object.
SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = 'service-account-credentials.json'
from google.oauth2 import service_account
cred = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
Google OAuth 2.0 Credentials:
These credentials are derived from a full OAuth 2.0 flow. These credentials are generated when your browser is launched to access Google Accounts for authorizing access. This process is much more complicated and requires a fair amount of code to implement and requires a built-in web server for the callback for authorization.
This method provides additional features such as being able to run everything in a browser, example you can create a Cloud Storage File Browser, but be careful that you understand the security implications. This method is the technique used to support Google Sign-In, etc. I like to use this method to authenticate users before allowing posting on websites, etc. The possibilities are endless with correctly authorized OAuth 2.0 identities and scopes.
Example code using google_auth_oauthlib
:
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file(
'client_secrets.json',
scopes=scope)
cred = flow.run_local_server(
host='localhost',
port=8088,
authorization_prompt_message='Please visit this URL: {url}',
success_message='The auth flow is complete; you may close this window.',
open_browser=True)
Example code using the requests_oauthlib
library:
from requests_oauthlib import OAuth2Session
gcp = OAuth2Session(
app.config['gcp_client_id'],
scope=scope,
redirect_uri=redirect_uri)
# print('Requesting authorization url:', authorization_base_url)
authorization_url, state = gcp.authorization_url(
authorization_base_url,
access_type="offline",
prompt="consent",
include_granted_scopes='true')
session['oauth_state'] = state
return redirect(authorization_url)
# Next section of code after the browser approves the request
token = gcp.fetch_token(
token_url,
client_secret=app.config['gcp_client_secret'],
authorization_response=request.url)
I additionally came across this github example: github.com/GoogleCloudPlatform/python-docs-samples/blob/master/… How does this relate to your above mentioned discussion?
– indraniel
Nov 26 '18 at 0:32
This example obtains the access token from the VM Instance metadata. This would be equivalent to the code in my answer under Compute Enginecred = compute_engine.Credentials()
. This example shows how to access the metadata server directly which is also important to know / understand.
– John Hanley
Nov 26 '18 at 0:37
If you are interested in further reading about Google Cloud Credentials, I have written a bunch of articles on my website: jhanley.com
– John Hanley
Nov 26 '18 at 0:40
Thanks again for the explanations and link to further readings!
– indraniel
Nov 26 '18 at 0:45
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53472429%2fhow-to-get-a-gcp-bearer-token-programmatically-with-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The answer depends on your environment and how you want to create / obtain credentials.
What are Google Cloud Credentials?
Google Cloud credentials are an OAuth 2.0 token. This token has at a minimum an Access Token
and optionally a Refresh Token
, Client ID Token
, and supporting parameters such as expiration
, Service Account Email
or Client Email
, etc.
The important item in Google Cloud APIs is the Access Token
. This token is what authorizes access to the cloud. This token can be used in programs such as curl
, software such as python
, etc and does not require an SDK. The Access Token
is used in the HTTP Authorization
header.
What is an Access Token?
An access token is an opaque value generated by Google that is derived from a Signed JWT, more correctly called JWS. A JWT consists of a header and claims (the payload) Json structures. These two Json structures are signed with the Service Account's Private Key. These values are base64 encoded and concatenated to create the Access Key.
The format of an Access Token is: base64(header) + '.' + base64(payload) + '.' + base64(signature)
.
Here is an example JWT:
Header:
{
"alg": "RS256",
"typ": "JWT",
"kid": "42ba1e234ac91ffca687a5b5b3d0ca2d7ce0fc0a"
}
Payload:
{
"iss": "myservice@myproject.iam.gserviceaccount.com",
"iat": 1493833746,
"aud": "myservice.appspot.com",
"exp": 1493837346,
"sub": "myservice@myproject.iam.gserviceaccount.com"
}
Using an Access Token:
Example that will start a VM instance. Replace PROJECT_ID, ZONE and INSTANCE_NAME. This example is for Windows.
curl -v -X GET -H "Authorization: Bearer <access_token_here>" ^
https://www.googleapis.com/compute/v1/projects/%PROJECT_ID%/zones/%ZONE%/instances/%INSTANCE_NAME%/start
Compute Engine Service Account:
Dustin's answer is correct for this case, but I will include for completeness with some additional information.
These credentials are automatically created for you by GCP and are obtained from the VM Instance metadata. Permissions are controlled by Cloud API access scopes
in the Google Console.
However, these credentials have some limitations. To modify the credentials you must stop the VM Instance first. Additionally, not all permissions (roles) are supported.
from google.auth import compute_engine
cred = compute_engine.Credentials()
Service Account Credentials:
Until you understand all of the types of credentials and their use cases, these are the credentials that you will use for everything except for gcloud
and gsutil
. Understanding these credentials will make working with Google Cloud much simpler when writing programs. Obtaining credentials from a Google Service Account Json file is easy. The only item to make note of is that credentials expire (typically 60 minutes) and either need to be refreshed or recreated.
gcloud auth print-access-token
is NOT recommended. Service Account Credentials are the recommended method by Google.
These credentials are created by the Console, gcloud or via programs / APIs. Permissions are assigned to the creditials by IAM and function inside Compute Engine, App Engine, Firestore, Kubernetes, etc. as well as other environments outside of Google Cloud. These credentials are downloaded from Google Cloud and stored in a Json file. Notice the scopes
parameter. This defines permissions that are granted to the resulting credentials object.
SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = 'service-account-credentials.json'
from google.oauth2 import service_account
cred = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
Google OAuth 2.0 Credentials:
These credentials are derived from a full OAuth 2.0 flow. These credentials are generated when your browser is launched to access Google Accounts for authorizing access. This process is much more complicated and requires a fair amount of code to implement and requires a built-in web server for the callback for authorization.
This method provides additional features such as being able to run everything in a browser, example you can create a Cloud Storage File Browser, but be careful that you understand the security implications. This method is the technique used to support Google Sign-In, etc. I like to use this method to authenticate users before allowing posting on websites, etc. The possibilities are endless with correctly authorized OAuth 2.0 identities and scopes.
Example code using google_auth_oauthlib
:
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file(
'client_secrets.json',
scopes=scope)
cred = flow.run_local_server(
host='localhost',
port=8088,
authorization_prompt_message='Please visit this URL: {url}',
success_message='The auth flow is complete; you may close this window.',
open_browser=True)
Example code using the requests_oauthlib
library:
from requests_oauthlib import OAuth2Session
gcp = OAuth2Session(
app.config['gcp_client_id'],
scope=scope,
redirect_uri=redirect_uri)
# print('Requesting authorization url:', authorization_base_url)
authorization_url, state = gcp.authorization_url(
authorization_base_url,
access_type="offline",
prompt="consent",
include_granted_scopes='true')
session['oauth_state'] = state
return redirect(authorization_url)
# Next section of code after the browser approves the request
token = gcp.fetch_token(
token_url,
client_secret=app.config['gcp_client_secret'],
authorization_response=request.url)
I additionally came across this github example: github.com/GoogleCloudPlatform/python-docs-samples/blob/master/… How does this relate to your above mentioned discussion?
– indraniel
Nov 26 '18 at 0:32
This example obtains the access token from the VM Instance metadata. This would be equivalent to the code in my answer under Compute Enginecred = compute_engine.Credentials()
. This example shows how to access the metadata server directly which is also important to know / understand.
– John Hanley
Nov 26 '18 at 0:37
If you are interested in further reading about Google Cloud Credentials, I have written a bunch of articles on my website: jhanley.com
– John Hanley
Nov 26 '18 at 0:40
Thanks again for the explanations and link to further readings!
– indraniel
Nov 26 '18 at 0:45
add a comment |
The answer depends on your environment and how you want to create / obtain credentials.
What are Google Cloud Credentials?
Google Cloud credentials are an OAuth 2.0 token. This token has at a minimum an Access Token
and optionally a Refresh Token
, Client ID Token
, and supporting parameters such as expiration
, Service Account Email
or Client Email
, etc.
The important item in Google Cloud APIs is the Access Token
. This token is what authorizes access to the cloud. This token can be used in programs such as curl
, software such as python
, etc and does not require an SDK. The Access Token
is used in the HTTP Authorization
header.
What is an Access Token?
An access token is an opaque value generated by Google that is derived from a Signed JWT, more correctly called JWS. A JWT consists of a header and claims (the payload) Json structures. These two Json structures are signed with the Service Account's Private Key. These values are base64 encoded and concatenated to create the Access Key.
The format of an Access Token is: base64(header) + '.' + base64(payload) + '.' + base64(signature)
.
Here is an example JWT:
Header:
{
"alg": "RS256",
"typ": "JWT",
"kid": "42ba1e234ac91ffca687a5b5b3d0ca2d7ce0fc0a"
}
Payload:
{
"iss": "myservice@myproject.iam.gserviceaccount.com",
"iat": 1493833746,
"aud": "myservice.appspot.com",
"exp": 1493837346,
"sub": "myservice@myproject.iam.gserviceaccount.com"
}
Using an Access Token:
Example that will start a VM instance. Replace PROJECT_ID, ZONE and INSTANCE_NAME. This example is for Windows.
curl -v -X GET -H "Authorization: Bearer <access_token_here>" ^
https://www.googleapis.com/compute/v1/projects/%PROJECT_ID%/zones/%ZONE%/instances/%INSTANCE_NAME%/start
Compute Engine Service Account:
Dustin's answer is correct for this case, but I will include for completeness with some additional information.
These credentials are automatically created for you by GCP and are obtained from the VM Instance metadata. Permissions are controlled by Cloud API access scopes
in the Google Console.
However, these credentials have some limitations. To modify the credentials you must stop the VM Instance first. Additionally, not all permissions (roles) are supported.
from google.auth import compute_engine
cred = compute_engine.Credentials()
Service Account Credentials:
Until you understand all of the types of credentials and their use cases, these are the credentials that you will use for everything except for gcloud
and gsutil
. Understanding these credentials will make working with Google Cloud much simpler when writing programs. Obtaining credentials from a Google Service Account Json file is easy. The only item to make note of is that credentials expire (typically 60 minutes) and either need to be refreshed or recreated.
gcloud auth print-access-token
is NOT recommended. Service Account Credentials are the recommended method by Google.
These credentials are created by the Console, gcloud or via programs / APIs. Permissions are assigned to the creditials by IAM and function inside Compute Engine, App Engine, Firestore, Kubernetes, etc. as well as other environments outside of Google Cloud. These credentials are downloaded from Google Cloud and stored in a Json file. Notice the scopes
parameter. This defines permissions that are granted to the resulting credentials object.
SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = 'service-account-credentials.json'
from google.oauth2 import service_account
cred = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
Google OAuth 2.0 Credentials:
These credentials are derived from a full OAuth 2.0 flow. These credentials are generated when your browser is launched to access Google Accounts for authorizing access. This process is much more complicated and requires a fair amount of code to implement and requires a built-in web server for the callback for authorization.
This method provides additional features such as being able to run everything in a browser, example you can create a Cloud Storage File Browser, but be careful that you understand the security implications. This method is the technique used to support Google Sign-In, etc. I like to use this method to authenticate users before allowing posting on websites, etc. The possibilities are endless with correctly authorized OAuth 2.0 identities and scopes.
Example code using google_auth_oauthlib
:
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file(
'client_secrets.json',
scopes=scope)
cred = flow.run_local_server(
host='localhost',
port=8088,
authorization_prompt_message='Please visit this URL: {url}',
success_message='The auth flow is complete; you may close this window.',
open_browser=True)
Example code using the requests_oauthlib
library:
from requests_oauthlib import OAuth2Session
gcp = OAuth2Session(
app.config['gcp_client_id'],
scope=scope,
redirect_uri=redirect_uri)
# print('Requesting authorization url:', authorization_base_url)
authorization_url, state = gcp.authorization_url(
authorization_base_url,
access_type="offline",
prompt="consent",
include_granted_scopes='true')
session['oauth_state'] = state
return redirect(authorization_url)
# Next section of code after the browser approves the request
token = gcp.fetch_token(
token_url,
client_secret=app.config['gcp_client_secret'],
authorization_response=request.url)
I additionally came across this github example: github.com/GoogleCloudPlatform/python-docs-samples/blob/master/… How does this relate to your above mentioned discussion?
– indraniel
Nov 26 '18 at 0:32
This example obtains the access token from the VM Instance metadata. This would be equivalent to the code in my answer under Compute Enginecred = compute_engine.Credentials()
. This example shows how to access the metadata server directly which is also important to know / understand.
– John Hanley
Nov 26 '18 at 0:37
If you are interested in further reading about Google Cloud Credentials, I have written a bunch of articles on my website: jhanley.com
– John Hanley
Nov 26 '18 at 0:40
Thanks again for the explanations and link to further readings!
– indraniel
Nov 26 '18 at 0:45
add a comment |
The answer depends on your environment and how you want to create / obtain credentials.
What are Google Cloud Credentials?
Google Cloud credentials are an OAuth 2.0 token. This token has at a minimum an Access Token
and optionally a Refresh Token
, Client ID Token
, and supporting parameters such as expiration
, Service Account Email
or Client Email
, etc.
The important item in Google Cloud APIs is the Access Token
. This token is what authorizes access to the cloud. This token can be used in programs such as curl
, software such as python
, etc and does not require an SDK. The Access Token
is used in the HTTP Authorization
header.
What is an Access Token?
An access token is an opaque value generated by Google that is derived from a Signed JWT, more correctly called JWS. A JWT consists of a header and claims (the payload) Json structures. These two Json structures are signed with the Service Account's Private Key. These values are base64 encoded and concatenated to create the Access Key.
The format of an Access Token is: base64(header) + '.' + base64(payload) + '.' + base64(signature)
.
Here is an example JWT:
Header:
{
"alg": "RS256",
"typ": "JWT",
"kid": "42ba1e234ac91ffca687a5b5b3d0ca2d7ce0fc0a"
}
Payload:
{
"iss": "myservice@myproject.iam.gserviceaccount.com",
"iat": 1493833746,
"aud": "myservice.appspot.com",
"exp": 1493837346,
"sub": "myservice@myproject.iam.gserviceaccount.com"
}
Using an Access Token:
Example that will start a VM instance. Replace PROJECT_ID, ZONE and INSTANCE_NAME. This example is for Windows.
curl -v -X GET -H "Authorization: Bearer <access_token_here>" ^
https://www.googleapis.com/compute/v1/projects/%PROJECT_ID%/zones/%ZONE%/instances/%INSTANCE_NAME%/start
Compute Engine Service Account:
Dustin's answer is correct for this case, but I will include for completeness with some additional information.
These credentials are automatically created for you by GCP and are obtained from the VM Instance metadata. Permissions are controlled by Cloud API access scopes
in the Google Console.
However, these credentials have some limitations. To modify the credentials you must stop the VM Instance first. Additionally, not all permissions (roles) are supported.
from google.auth import compute_engine
cred = compute_engine.Credentials()
Service Account Credentials:
Until you understand all of the types of credentials and their use cases, these are the credentials that you will use for everything except for gcloud
and gsutil
. Understanding these credentials will make working with Google Cloud much simpler when writing programs. Obtaining credentials from a Google Service Account Json file is easy. The only item to make note of is that credentials expire (typically 60 minutes) and either need to be refreshed or recreated.
gcloud auth print-access-token
is NOT recommended. Service Account Credentials are the recommended method by Google.
These credentials are created by the Console, gcloud or via programs / APIs. Permissions are assigned to the creditials by IAM and function inside Compute Engine, App Engine, Firestore, Kubernetes, etc. as well as other environments outside of Google Cloud. These credentials are downloaded from Google Cloud and stored in a Json file. Notice the scopes
parameter. This defines permissions that are granted to the resulting credentials object.
SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = 'service-account-credentials.json'
from google.oauth2 import service_account
cred = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
Google OAuth 2.0 Credentials:
These credentials are derived from a full OAuth 2.0 flow. These credentials are generated when your browser is launched to access Google Accounts for authorizing access. This process is much more complicated and requires a fair amount of code to implement and requires a built-in web server for the callback for authorization.
This method provides additional features such as being able to run everything in a browser, example you can create a Cloud Storage File Browser, but be careful that you understand the security implications. This method is the technique used to support Google Sign-In, etc. I like to use this method to authenticate users before allowing posting on websites, etc. The possibilities are endless with correctly authorized OAuth 2.0 identities and scopes.
Example code using google_auth_oauthlib
:
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file(
'client_secrets.json',
scopes=scope)
cred = flow.run_local_server(
host='localhost',
port=8088,
authorization_prompt_message='Please visit this URL: {url}',
success_message='The auth flow is complete; you may close this window.',
open_browser=True)
Example code using the requests_oauthlib
library:
from requests_oauthlib import OAuth2Session
gcp = OAuth2Session(
app.config['gcp_client_id'],
scope=scope,
redirect_uri=redirect_uri)
# print('Requesting authorization url:', authorization_base_url)
authorization_url, state = gcp.authorization_url(
authorization_base_url,
access_type="offline",
prompt="consent",
include_granted_scopes='true')
session['oauth_state'] = state
return redirect(authorization_url)
# Next section of code after the browser approves the request
token = gcp.fetch_token(
token_url,
client_secret=app.config['gcp_client_secret'],
authorization_response=request.url)
The answer depends on your environment and how you want to create / obtain credentials.
What are Google Cloud Credentials?
Google Cloud credentials are an OAuth 2.0 token. This token has at a minimum an Access Token
and optionally a Refresh Token
, Client ID Token
, and supporting parameters such as expiration
, Service Account Email
or Client Email
, etc.
The important item in Google Cloud APIs is the Access Token
. This token is what authorizes access to the cloud. This token can be used in programs such as curl
, software such as python
, etc and does not require an SDK. The Access Token
is used in the HTTP Authorization
header.
What is an Access Token?
An access token is an opaque value generated by Google that is derived from a Signed JWT, more correctly called JWS. A JWT consists of a header and claims (the payload) Json structures. These two Json structures are signed with the Service Account's Private Key. These values are base64 encoded and concatenated to create the Access Key.
The format of an Access Token is: base64(header) + '.' + base64(payload) + '.' + base64(signature)
.
Here is an example JWT:
Header:
{
"alg": "RS256",
"typ": "JWT",
"kid": "42ba1e234ac91ffca687a5b5b3d0ca2d7ce0fc0a"
}
Payload:
{
"iss": "myservice@myproject.iam.gserviceaccount.com",
"iat": 1493833746,
"aud": "myservice.appspot.com",
"exp": 1493837346,
"sub": "myservice@myproject.iam.gserviceaccount.com"
}
Using an Access Token:
Example that will start a VM instance. Replace PROJECT_ID, ZONE and INSTANCE_NAME. This example is for Windows.
curl -v -X GET -H "Authorization: Bearer <access_token_here>" ^
https://www.googleapis.com/compute/v1/projects/%PROJECT_ID%/zones/%ZONE%/instances/%INSTANCE_NAME%/start
Compute Engine Service Account:
Dustin's answer is correct for this case, but I will include for completeness with some additional information.
These credentials are automatically created for you by GCP and are obtained from the VM Instance metadata. Permissions are controlled by Cloud API access scopes
in the Google Console.
However, these credentials have some limitations. To modify the credentials you must stop the VM Instance first. Additionally, not all permissions (roles) are supported.
from google.auth import compute_engine
cred = compute_engine.Credentials()
Service Account Credentials:
Until you understand all of the types of credentials and their use cases, these are the credentials that you will use for everything except for gcloud
and gsutil
. Understanding these credentials will make working with Google Cloud much simpler when writing programs. Obtaining credentials from a Google Service Account Json file is easy. The only item to make note of is that credentials expire (typically 60 minutes) and either need to be refreshed or recreated.
gcloud auth print-access-token
is NOT recommended. Service Account Credentials are the recommended method by Google.
These credentials are created by the Console, gcloud or via programs / APIs. Permissions are assigned to the creditials by IAM and function inside Compute Engine, App Engine, Firestore, Kubernetes, etc. as well as other environments outside of Google Cloud. These credentials are downloaded from Google Cloud and stored in a Json file. Notice the scopes
parameter. This defines permissions that are granted to the resulting credentials object.
SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = 'service-account-credentials.json'
from google.oauth2 import service_account
cred = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
Google OAuth 2.0 Credentials:
These credentials are derived from a full OAuth 2.0 flow. These credentials are generated when your browser is launched to access Google Accounts for authorizing access. This process is much more complicated and requires a fair amount of code to implement and requires a built-in web server for the callback for authorization.
This method provides additional features such as being able to run everything in a browser, example you can create a Cloud Storage File Browser, but be careful that you understand the security implications. This method is the technique used to support Google Sign-In, etc. I like to use this method to authenticate users before allowing posting on websites, etc. The possibilities are endless with correctly authorized OAuth 2.0 identities and scopes.
Example code using google_auth_oauthlib
:
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file(
'client_secrets.json',
scopes=scope)
cred = flow.run_local_server(
host='localhost',
port=8088,
authorization_prompt_message='Please visit this URL: {url}',
success_message='The auth flow is complete; you may close this window.',
open_browser=True)
Example code using the requests_oauthlib
library:
from requests_oauthlib import OAuth2Session
gcp = OAuth2Session(
app.config['gcp_client_id'],
scope=scope,
redirect_uri=redirect_uri)
# print('Requesting authorization url:', authorization_base_url)
authorization_url, state = gcp.authorization_url(
authorization_base_url,
access_type="offline",
prompt="consent",
include_granted_scopes='true')
session['oauth_state'] = state
return redirect(authorization_url)
# Next section of code after the browser approves the request
token = gcp.fetch_token(
token_url,
client_secret=app.config['gcp_client_secret'],
authorization_response=request.url)
edited Nov 25 '18 at 23:53
answered Nov 25 '18 at 23:02
John HanleyJohn Hanley
17.1k2633
17.1k2633
I additionally came across this github example: github.com/GoogleCloudPlatform/python-docs-samples/blob/master/… How does this relate to your above mentioned discussion?
– indraniel
Nov 26 '18 at 0:32
This example obtains the access token from the VM Instance metadata. This would be equivalent to the code in my answer under Compute Enginecred = compute_engine.Credentials()
. This example shows how to access the metadata server directly which is also important to know / understand.
– John Hanley
Nov 26 '18 at 0:37
If you are interested in further reading about Google Cloud Credentials, I have written a bunch of articles on my website: jhanley.com
– John Hanley
Nov 26 '18 at 0:40
Thanks again for the explanations and link to further readings!
– indraniel
Nov 26 '18 at 0:45
add a comment |
I additionally came across this github example: github.com/GoogleCloudPlatform/python-docs-samples/blob/master/… How does this relate to your above mentioned discussion?
– indraniel
Nov 26 '18 at 0:32
This example obtains the access token from the VM Instance metadata. This would be equivalent to the code in my answer under Compute Enginecred = compute_engine.Credentials()
. This example shows how to access the metadata server directly which is also important to know / understand.
– John Hanley
Nov 26 '18 at 0:37
If you are interested in further reading about Google Cloud Credentials, I have written a bunch of articles on my website: jhanley.com
– John Hanley
Nov 26 '18 at 0:40
Thanks again for the explanations and link to further readings!
– indraniel
Nov 26 '18 at 0:45
I additionally came across this github example: github.com/GoogleCloudPlatform/python-docs-samples/blob/master/… How does this relate to your above mentioned discussion?
– indraniel
Nov 26 '18 at 0:32
I additionally came across this github example: github.com/GoogleCloudPlatform/python-docs-samples/blob/master/… How does this relate to your above mentioned discussion?
– indraniel
Nov 26 '18 at 0:32
This example obtains the access token from the VM Instance metadata. This would be equivalent to the code in my answer under Compute Engine
cred = compute_engine.Credentials()
. This example shows how to access the metadata server directly which is also important to know / understand.– John Hanley
Nov 26 '18 at 0:37
This example obtains the access token from the VM Instance metadata. This would be equivalent to the code in my answer under Compute Engine
cred = compute_engine.Credentials()
. This example shows how to access the metadata server directly which is also important to know / understand.– John Hanley
Nov 26 '18 at 0:37
If you are interested in further reading about Google Cloud Credentials, I have written a bunch of articles on my website: jhanley.com
– John Hanley
Nov 26 '18 at 0:40
If you are interested in further reading about Google Cloud Credentials, I have written a bunch of articles on my website: jhanley.com
– John Hanley
Nov 26 '18 at 0:40
Thanks again for the explanations and link to further readings!
– indraniel
Nov 26 '18 at 0:45
Thanks again for the explanations and link to further readings!
– indraniel
Nov 26 '18 at 0:45
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53472429%2fhow-to-get-a-gcp-bearer-token-programmatically-with-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Do you want to get this in a GCP environment like Compute Engine, App Engine, Cloud Functions, etc? Or outside GCP, like on your local machine?
– Dustin Ingram
Nov 25 '18 at 22:10
I'm building an app/script inside a GCP Compute Engine instance with a service account. That's my first preference. I'm also interested to know how it would be like if it was run outside GCP on my local machine too.
– indraniel
Nov 25 '18 at 22:13