Friday, August 7, 2015

Google API Uploading Files

See also http://www.sizustech.blogspot.com/2015/08/getting-started-with-google-drive-api.html on how to generate Auth Tokens. You will need that for this next section as Auth Tokens will be referenced using ${AuthToken}

Upload Dummy File (curl)

Open Git Bash and execute the following command after replacing the Auth Token

curl -k -H "Content-Type: plain/text" -H "Authorization: Bearer ${AuthToken}" -D- -X POST --data "Test adding this content into a new file" https://www.googleapis.com/upload/drive/v2/files?uploadType=media > testUploadResponse.txt

For example, the command might look like this...
curl -k -H "Content-Type: plain/text" -H "Authorization: Bearer ya29.yAFLSqN1xaR9EKuMrcJu6iLMt9PyaRqotTK01PbaPjd7gZZMtqE9XgFgHz3A5IEwKV7-" -D- -X POST --data "Test adding this content into a new file" https://www.googleapis.com/upload/drive/v2/files?uploadType=media > testUploadResponse.txt

Go to http://www.google.com/drive under the account for which the Auth Token was created and search for a file Untitled. If you open this file in a text editor like Notepad, you will see the contents inside the file as "Test adding this content into a new file"

Upload Dummy File (Java)

import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
import org.apache.http.entity.StringEntity;
import org.json.JSONObject;

// Step 2 - UPLOAD THE FIRST FILE
// curl -k -H "Content-Type: plain/text" -H "Authorization: Bearer ya29.yAFLSqN1xaR9EKuMrcJu6iLMt9PyaRqotTK01PbaPjd7gZZMtqE9XgFgHz3A5IEwKV7-" -D- -X POST --data "Test adding this content into a new file" https://www.googleapis.com/upload/drive/v2/files?uploadType=media > testUploadResponse.txt
public class UploadTheFirstFile {
    public static void main(String[] args) {
        try {
            String token = "ya29.yAFLSqN1xaR9EKuMrcJu6iLMt9PyaRqotTK01PbaPjd7gZZMtqE9XgFgHz3A5IEwKV7-";
            HttpPost httpPost = new HttpPost("https://www.googleapis.com/upload/drive/v2/files?uploadType=media");
            httpPost.addHeader("Authorization", "Bearer "+token);
            httpPost.addHeader("Content-Type", "plain/text");
        
            StringEntity entity = new StringEntity("Test adding this content into a new file");
            httpPost.setEntity(entity);
            
            CloseableHttpResponse response = new DefaultHttpClient().execute(httpPost);
            String jsonResponse = EntityUtils.toString(response.getEntity());
            response.close();
            System.out.println("jsonResponse:"+jsonResponse);
            
            JSONObject jsonObject = new JSONObject(jsonResponse);
            String id = jsonObject.getString("id");
            System.out.println("Created resource... ID: "+id);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

Create a Folder for your Uploads (curl)

You can parse the JSON response to find the id for an existing folder.  However, for this example, we will create a folder for storage call "My Upload Folder".

curl -k -H "Authorization: Bearer {$AuthToken}" -H "Content-Type:application/json" --data '{"title":"My Upload Folder","mimeType":"application/vnd.google-apps.folder"}' https://www.googleapis.com/drive/v2/files > folderResponse.txt

The JSON response will contain the folder id, title and mimeType. You can look in the folderResponse.txt file for these.

{ ... "id": "0B9pC9EpixPYFNk9WMWtmT1duQUk" ... "title": "My Upload Folder" ... "mimeType": "application/vnd.google-apps.folder" ... }

Note: We will use this folder ID in this blog post as {$folderID}.

Create a Folder for your Uploads (Java)

import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
import org.apache.http.entity.StringEntity;
import org.json.JSONObject;

// curl -k -H "Authorization: Bearer {$AuthToken}" -H "Content-Type:application/json" --data '{"title":"My Upload Folder","mimeType":"application/vnd.google-apps.folder"}' https://www.googleapis.com/drive/v2/files > folderResponse.txt
public class CreateTheFirstFolder {
    public static void main(String[] args) {
        try {
            String token = "ya29.yAGZ79jCXHvo8jSGOIBgT3Rxbk6Ot1ca1kh6zF_qfpwbnlDF4gdwbN-KCJR0hKxBXo81";
            HttpPost httpPost = new HttpPost("https://www.googleapis.com/drive/v2/files");
            httpPost.addHeader("Authorization", "Bearer "+token);
            httpPost.addHeader("Content-Type", "application/json");
        
            JSONObject json = new JSONObject();
            json.put("title", "My Upload Folder");
            json.put("mimeType", "application/vnd.google-apps.folder");
            
            StringEntity entity = new StringEntity(json.toString());
            httpPost.setEntity(entity);
            
            CloseableHttpResponse response = new DefaultHttpClient().execute(httpPost);
            String jsonResponse = EntityUtils.toString(response.getEntity());
            response.close();
            System.out.println("jsonResponse:"+jsonResponse);
            
            JSONObject jsonObject = new JSONObject(jsonResponse);
            String id = jsonObject.getString("id");
            System.out.println("Created resource... ID: "+id);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

The output was

Created resource... ID: 0B9pC9EpixPYFNk9WMWtmT1duQUk

Note: We will use this folder ID in this blog post as {$folderID}.

Upload a Text File (curl)

Now, we will add a title, provide the folder using the parent key and provide the content in a single curl command.  This will require creating a multi part request.

curl -k -H "Authorization: Bearer ${AuthToken}" -H 'Content-Type: multipart/form-data; boundary="foo_bar_baz"' --data $'--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{"title": "My File.txt","parents":[{"kind":"drive#fileLink","id":"${folderID}"}]}\r\n--foo_bar_baz\r\nContent-Type: plain/text\r\n\r\nThis is in the file\r\n--foo_bar_baz--' https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart

curl -k -H "Authorization: Bearer ya29.yAG2xVUntrdwwY9DnwiWeRme6A6unAMDr1uiouTekpV5ee6LS9UnM9iISB-GGNJGZK7b" -H 'Content-Type: multipart/form-data; boundary="foo_bar_baz"' --data $'--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{"title": "My File.txt","parents":[{"kind":"drive#fileLink","id":"0B9pC9EpixPYFNk9WMWtmT1duQUk"}]}\r\n--foo_bar_baz\r\nContent-Type: plain/text\r\n\r\nThis is in the file\r\n--foo_bar_baz--' https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart

The boundaries of the multi parts are marked with --foo_bar_baz and the spaces "\r\n" have to be exactly as is otherwise you will get an error "Missing end boundary in multi part body".  You also have to use the $ character to make sure these are processed as expected.

Upload a Text File (curl) - Stitching Together a Multipart Request

This is example is used to give an easy example of stitching together a mutli part form.  This is useful for stitching together files.  Note that the cat command in linux will prepend a "\n", so if you look closely, one of the "\n" characters is removed.

Below, you will have to add replace the ${AuthToken} and the ${folderID}.

touch txtDataFile.txt

echo $'--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{"title": "My File.txt","parents":[{"kind":"drive#fileLink","id":"${folderID}"}]}\r\n--foo_bar_baz\r\nContent-Type: plain/text\r\n\r' >> txtDataFile.txt

cat TestUpload.txt >> txtDataFile.txt

echo $'\r\n--foo_bar_baz--' >> txtDataFile.txt

curl -k -H "Authorization: Bearer ${AuthToken}" -H 'Content-Type: multipart/form-data; boundary="foo_bar_baz"' --data-binary "@txtDataFile.txt" https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart

For example...

touch txtDataFile.txt

echo $'--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{"title": "My File.txt","parents":[{"kind":"drive#fileLink","id":"0B9pC9EpixPYFNk9WMWtmT1duQUk"}]}\r\n--foo_bar_baz\r\nContent-Type: plain/text\r\n\r' >> txtDataFile.txt

cat TestUpload.txt >> txtDataFile.txt

echo $'\r\n--foo_bar_baz--' >> txtDataFile.txt

curl -k -H "Authorization: Bearer ya29.xwHjsm--zQ6Wi5qD6mAoMfnDcNdCl0FvaUN8TuAwCvMJ4-YYq2ozj1vKCE1aAGECW4s8" -H 'Content-Type: multipart/form-data; boundary="foo_bar_baz"' --data-binary "@txtDataFile.txt" https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart

Upload a Text File (Java)

import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
import org.apache.http.entity.StringEntity;
import org.json.JSONArray;
import org.json.JSONObject;

// curl -k -H "Authorization: Bearer ya29.xwHjsm--zQ6Wi5qD6mAoMfnDcNdCl0FvaUN8TuAwCvMJ4-YYq2ozj1vKCE1aAGECW4s8" -H 'Content-Type: multipart/form-data; boundary="foo_bar_baz"' --data $'--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{"title": "My File.txt","parents":[{"kind":"drive#fileLink","id":"0B9pC9EpixPYFNk9WMWtmT1duQUk"}]}\r\n--foo_bar_baz\r\nContent-Type: plain/text\r\n\r\nThis is in the file\r\n--foo_bar_baz--' https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart
//
// touch txtDataFile.txt
// echo $'--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{"title": "My File.txt","parents":[{"kind":"drive#fileLink","id":"0B9pC9EpixPYFNk9WMWtmT1duQUk"}]}\r\n--foo_bar_baz\r\nContent-Type: plain/text\r\n\r' >> txtDataFile.txt
// cat TestUpload.txt >> txtDataFile.txt
// echo $'\r\n--foo_bar_baz--' >> txtDataFile.txt
// curl -k -H "Authorization: Bearer ya29.xwHjsm--zQ6Wi5qD6mAoMfnDcNdCl0FvaUN8TuAwCvMJ4-YYq2ozj1vKCE1aAGECW4s8" -H 'Content-Type: multipart/form-data; boundary="foo_bar_baz"' --data-binary "@txtDataFile.txt" https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart
public class CreateADataFileToUploadATextFile {
    public static void main(String[] args) {
        try {
            String accessToken = "ya29.xwHjsm--zQ6Wi5qD6mAoMfnDcNdCl0FvaUN8TuAwCvMJ4-YYq2ozj1vKCE1aAGECW4s8";
            HttpPost httpPost = new HttpPost("https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart");
            httpPost.addHeader("Authorization", "Bearer "+accessToken);
            httpPost.setHeader("Content-Type","multipart/related; boundary=foo_bar_baz");

            JSONObject json = new JSONObject();
            json.put("title", "My File.txt");
            JSONArray parents = new JSONArray();
            JSONObject parent = new JSONObject();
            parent.put("kind", "drive#fileLink");
            parent.put("id", "0B9pC9EpixPYFNk9WMWtmT1duQUk");
            parents.put(parent);
            json.put("parents", parents);
            
            String fileContent = "Contents of Test Upload File";
            
            StringEntity str=new StringEntity("--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n"+json.toString()+"\r\n--foo_bar_baz\r\nContent-Type: plain/text\r\n\r\n"+fileContent+"\r\n--foo_bar_baz--");
            httpPost.setEntity(str);
            
            CloseableHttpResponse response = new DefaultHttpClient().execute(httpPost);
            String jsonResponse = EntityUtils.toString(response.getEntity());
            response.close();
            System.out.println("jsonResponse:"+jsonResponse);
            
            JSONObject jsonObject = new JSONObject(jsonResponse);
            String id = jsonObject.getString("id");
            System.out.println("Created resource... ID: "+id);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

Upload an Image File (curl)

The main difference between this and the last example is the data is larger and the Content-Type is image/jpeg instead of text/plain.

touch jpgDataFile.txt

echo $'--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{"title": "My JPG.jpg","parents":[{"kind":"drive#fileLink","id":"0B9pC9EpixPYFNk9WMWtmT1duQUk"}]}\r\n--foo_bar_baz\r\nContent-Type: image/jpeg\r\n\r' >> jpgDataFile.txt

cat TreeHouse15.jpg >> jpgDataFile.txt

echo $'\r\n--foo_bar_baz--' >> jpgDataFile.txt

curl -k -H "Authorization: Bearer ya29.yAG2xVUntrdwwY9DnwiWeRme6A6unAMDr1uiouTekpV5ee6LS9UnM9iISB-GGNJGZK7b" -H 'Content-Type: multipart/form-data; boundary="foo_bar_baz"' --data-binary "@jpgDataFile.txt" https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart

Uploading an Image Data URI (Java)

To make things a little more interesting, we will use a Data URI for this example.

import org.apache.commons.codec.binary.Base64;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
import org.apache.http.entity.StringEntity;
import org.json.JSONArray;
import org.json.JSONObject;

// rm jpgDataFile.txt
// touch jpgDataFile.txt
// echo $'--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{"title": "My JPG.jpg","parents":[{"kind":"drive#fileLink","id":"0B9pC9EpixPYFNk9WMWtmT1duQUk"}]}\r\n--foo_bar_baz\r\nContent-Type: image/jpeg\r\n\r' >> jpgDataFile.txt
// cat TreeHouse15.jpg >> jpgDataFile.txt
// echo $'\r\n--foo_bar_baz--' >> jpgDataFile.txt
// curl -k -H "Authorization: Bearer ya29.xwFXZu-7vgVrcLyfzd1dZOLy3zjoifxGd6-luVgCJw_o8rMNrIGoGqQLfoPhDjtzgs80" -H 'Content-Type: multipart/form-data; boundary="foo_bar_baz"' --data-binary "@jpgDataFile.txt" https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart
public class CreateADataFileToUploadAnImageFile {
    public static void main(String[] args) {
        try {
            String accessToken = "ya29.yAG2xVUntrdwwY9DnwiWeRme6A6unAMDr1uiouTekpV5ee6LS9UnM9iISB-GGNJGZK7b";
            HttpPost httpPost = new HttpPost("https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart");
            httpPost.addHeader("Authorization", "Bearer "+accessToken);
            httpPost.setHeader("Content-Type","multipart/related; boundary=foo_bar_baz");

            JSONObject json = new JSONObject();
            json.put("title", "My JPG.jpg");
            JSONArray parents = new JSONArray();
            JSONObject parent = new JSONObject();
            parent.put("kind", "drive#fileLink");
            parent.put("id", "0B9pC9EpixPYFNk9WMWtmT1duQUk");
            parents.put(parent);
            json.put("parents", parents);
            
            String dataURI = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAQMAAAAlPW0iAAAABlBMVEUAAAD///+l2Z/dAAAAM0lEQVR4nGP4/5/h/1+G/58ZDrAz3D/McH8yw83NDDeNGe4Ug9C9zwz3gVLMDA/A6P9/AFGGFyjOXZtQAAAAAElFTkSuQmCC";
            String encodingPrefix = "base64,";
            int contentStartIndex = dataURI.indexOf(encodingPrefix) + encodingPrefix.length();
            byte[] imageData = Base64.decodeBase64(dataURI.substring(contentStartIndex));
            String fileContent = new String(imageData);
            
            StringEntity str=new StringEntity("--foo_bar_baz\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n"+json.toString()+"\r\n--foo_bar_baz\r\nContent-Type: image/jpeg\r\n\r\n"+fileContent+"\r\n--foo_bar_baz--");
            httpPost.setEntity(str);
            
            CloseableHttpResponse response = new DefaultHttpClient().execute(httpPost);
            String jsonResponse = EntityUtils.toString(response.getEntity());
            response.close();
            System.out.println("jsonResponse:"+jsonResponse);
            
            JSONObject jsonObject = new JSONObject(jsonResponse);
            String id = jsonObject.getString("id");
            System.out.println("Created resource... ID: "+id);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}
Posted by at 1 comment:
Labels: , ,

Getting Started with the Google Drive API

Step 1. Create a Client ID 

See also Retrieve and Use OAuth 2.0 Credentials.
  • Go to https://console.developers.google.com/
  • Switch to appropriate Google account
  • Click on ENABLE APIS in the "Enable Google APIs for use in your apps
  • Named the project "Upload Project" and agreed to the terms
  • Click Drive API and hit Enable API
  • Click Credentials in left nav
  • Click Create new Client ID
  • Select Web Application
  • In the form for Consent Screen, name the product "Upload Project"
  • Click Save
  • Click Create Client ID
  • You will get "Client ID", "Client secret", "Redirect URIs" and "JavaScript origins"
Note: We will used ${clientID} to represent the client ID in this post.

Step 2. Generate a new Auth Token

For protocol information about OAuth protocol and workflow, see https://developers.google.com/identity/protocols/OAuth2?csw=1

For REST endpoint information on the oauth2 REST API used here, see https://developers.google.com/identity/protocols/OAuth2WebServer

For parameter information on Auth Token scopes for Google Drive API access, see https://developers.google.com/drive/v2/reference/files/insert

For parameter information on Auth Token scopes for You Tube API access, see https://developers.google.com/youtube/v3/guides/auth/client-side-web-apps
  • In developer console, edit settings and change Redirect URIs to http://localhost
  • The URL needed to generate the Auth Token looks something like this:
    • https://accounts.google.com/o/oauth2/auth?client_id=****nko08rm1s2qebr300ll3a0kakm3aue.apps.googleusercontent.com&redirect_uri=http://localhost&scope=https://www.googleapis.com/auth/drive&response_type=token
    • Open Google Chrome and enter this is the browser, after replacing ${clientID}
      • https://accounts.google.com/o/oauth2/auth?client_id=${clientID}&redirect_uri=http://localhost&scope=https://www.googleapis.com/auth/drive&response_type=token
    • Go through the flows to sign in (Username, Password, User Consent)
    • The URL in the Browser will contain the Google API access token which will be something like this:
      • http://localhost/#access_token=ya29.xwHjsm--zQ6Wi5qD6mAoMfnDcNdCl0FvaUN8TuAwCvMJ4-YYq2ozj1vKCE1aAGECW4s8&token_type=Bearer&expires_in=3600
    • The access token would then be something like this:
      • ya29.xwHjsm--zQ6Wi5qD6mAoMfnDcNdCl0FvaUN8TuAwCvMJ4-YYq2ozj1vKCE1aAGECW4s8
    Note: In this post, we will refer to this Auth Token using ${AuthToken}

    Step 3. List all files (curl)

    Open Git Bash command line and execute the following.  This will create a file filesResponse.txt with all the files in google drive.

    curl -k -H "Authorization: Bearer ${AuthToken}" https://www.googleapis.com/drive/v2/files > filesResponse.txt

    curl -k -H "Authorization: Bearer ya29.xwE3y4ErpmdEXVS3v02hC-DEm2gz05ZwpjOvrmdOXOcCSz0YALOH9A1Qc4YHQGnFPqjN" https://www.googleapis.com/drive/v2/files > filesResponse.txt

    You will get a json response with a ton of information.

    Alternate Step 3: List all files (Java)

    import org.apache.http.client.methods.CloseableHttpResponse;
    import org.apache.http.client.methods.HttpGet;
    import org.apache.http.impl.client.DefaultHttpClient;
    import org.apache.http.util.EntityUtils;
    // Step 1 - LIST ALL FILES TO TEST GOOGLE DRIVE API
    // curl -k -H "Authorization: Bearer ya29.xwE3y4ErpmdEXVS3v02hC-DEm2gz05ZwpjOvrmdOXOcCSz0YALOH9A1Qc4YHQGnFPqjN" https://www.googleapis.com/drive/v2/files > files.txt
    //<!-- Shipping with selenium-htmlunit-driver 2.41.0 -->
    //<!-- Includes httpclient 4.3.1 org.apache.http.client.methods.CloseableHttpResponse -->
    //<!-- Includes httpmime 4.3.1 org.apache.http.entity.mime -->
    //<!-- Includes commons-lang3 3.1 org.apache.commons.lang3.exception.ExceptionUtils -->
    //<dependency>
    //    <groupId>net.sourceforge.htmlunit</groupId>
    //    <artifactId>htmlunit</artifactId>
    //    <version>2.13</version>
    //</dependency>
    public class ListAllFiles {
        public static void main(String[] args) {
            try {
                String token = "ya29.yAFLSqN1xaR9EKuMrcJu6iLMt9PyaRqotTK01PbaPjd7gZZMtqE9XgFgHz3A5IEwKV7-";
                HttpGet httpGet = new HttpGet("https://www.googleapis.com/drive/v2/files");
                httpGet.addHeader("Authorization", "Bearer "+token);
                CloseableHttpResponse response = new DefaultHttpClient().execute(httpGet);
                String jsonResponse = EntityUtils.toString(response.getEntity());
                response.close();
                System.out.println("jsonResponse:"+jsonResponse);
            } catch (Exception ex) {
                ex.printStackTrace();
            }
        }
    }
    Posted by at No comments:
    Labels: ,
    Subscribe to: Posts (Atom)