Sunday, December 29, 2013

Testing authenticated GAE endpoints in Android Studio

Preface

This post assumes that we can already test unauthenticated GAE endpoints as described in Testing and debugging GAE Endpoints in Android Studio.  There we used an Intel Atom (x86) emulator for better performance.
To test authenticated GAE endpoints, we need a valid Google account.  To add a Google account, we need an emulator with Google APIs.  Unfortunately, Google APIs are not available for x86-based emulators and we have to revert to slower ARM-based emulators.  As an alternative and for better performance, we can test on real devices as described in From an emulator to a real device.

Create Google account

This is the easiest part.
  1. On a device or an ARM-based emulator with Google APIs, open Settings application
  2. Scroll down to Accounts section
  3. Tap on Add account
  4. Select Google
  5. Follow the Add a Google Account wizard

Add authentication support to GAE endpoints

This is a two-step process that is applied to Project-AppEngine project and is described in detail here.
First, we need to specify authorized clients in the API backend. Second, we need to add a user parameter to methods for auth.  Following conventions used in the earlier post, we've just modified Project-AppEngine/src/main/java/com/example/project/TestEndpoint.java.
These steps turn on GAE endpoints authentication. Re-deploy the project and verify that authentication is enforced by the backend.
  1. Re-deploy Project-AppEngine: appengine:update
  2. Go to APIs explorer for your application
  3. Verify that un-authenticated requests fail
  4. Turn on Authorize requests using OAuth 2.0
  5. Verity than authenticated requests work
Now we can create client applications that communicate with authenticated GAE endpoints. Details for Android clients are described here.  Since our goal is to test authenticated GAE endpoints, we need to do more work.

Enable authenticated endpoints testing

Even though we will be running instrumentation tests deployed on the local dev server, it is still required to add an OAuth test client to our GAE application.  Local dev server will not accept connections from un-registered clients.  If we don't complete this and the following steps, we will get the error below:
com.google.android.gms.auth.GoogleAuthException: Unknown

Specify the test package name (optional)

In earlier posts we used com.example.package as an application package.  Based on that, the default package name for Project-endpoints/src/instrumentTest is com.example.package.test.  To specify different package name, modify Project-endpoints/src/build.gradle:
android {
    // ...
    defaultConfig {
        // ...
        testPackageName ’some.package.name’
    }
}
We will use default name com.example.package.test.

Add a test client 

  1. Go back and login to Google Cloud Console
  2. Click on the project you are working with
  3. Click on APIs & auth
  4. Click on Credentials
  5. In the OAuth section, click Create New Client ID button
  6. Under Application Type chose Installed application
  7. Under Installed Application Type chose Android
  8. Enter com.example.package.test into the Package name field
  9. Into Signing certificate fingerprint field, enter the SHA1 code that was used for the application package.
  10. Click Create Client ID button.
Copy Client ID of the new application, it will be used in the next step.

Add test Client ID to endpoint's @Api

  1. Go back to Project-AppEngine/src/main/java/com/example/project/TestEndpoint.java. 
  2. Add the test Client ID to the clientIds parameter of the @Api annotation.
The annotated TestEndpoint should look something like this:
@Api(
    name = “testendpoint",
    namespace = @ApiNamespace(ownerDomain = "com.example”, 
        ownerName = "com.example", packagePath = “package”),
    scopes = { Consts.EMAIL_SCOPE },
    clientIds = { Consts.WEB_CLIENT_ID, Consts.ANDROID_CLIENT_ID, 
        Consts.ANDROID_CLIENT_ID_TEST },
    audiences = { Consts.ANDROID_AUDIENCE }
)
public class TestEndpoint {
    // ...
}
where Consts.ANDROID_CLIENT_ID_TEST is the Client ID of the test application from the previous step.

Re-deploy the project

Since we will be testing authenticated GAE endpoints on local dev server, this step is not really required.  However, for consistency, it is still better to re-deploy the project.  Just run appengine:update maven task to update the server backend.

Modify project configuration

Modify Project-endpoints/build.gradle

dependencies {
    // ...
    instrumentTestCompile 'com.google.android.gms:play-services:4.0.30’
    instrumentTestCompile 'com.google.http-client:google-http-client-gson:1.17.0-rc'
}
Here we added dependency on Google play services library.

Modify Project-endpoints/src/main/AndroidManifest.xml

Add
<meta-data android:name="com.google.android.gms.version"
 android:value="@integer/google_play_services_version"/>
under the <application/> tag.

Clean and re-build the project

If we don't clean/re-build the project, we will be running into run-time errors described in an earlier post.
  • Build -> Clean Project or
  • Build -> Rebuild Project

Modify setUp() method

This is the last step that is required to enable testing authenticated GAE endpoints.  
  1. Go to Project-endpoints/src/instrumentTest/java/com/example/project/TestEndpointTest.java
  2. Go to TestEndpointTest.setUp() method.  This is the only method that needs to be modified.
  3. Replace the last null parameter of the Testendpoint.Builder() constructor invocation with a valid credential parameter.
The final TestEndpointTest.setUp() should look something like below:
@Override
protected void setUp() throws Exception {
    super.setUp();
    final GoogleAccountCredential credential = 
        GoogleAccountCredential.usingAudience(getContext(), 
            String.format("server:client_id:%s", WEB_CLIENT_ID));
    credential.setSelectedAccountName(ACCOUNT_NAME);
    final Testendpoint.Builder bld = new Testendpoint.Builder(
        AndroidHttp.newCompatibleTransport(), new GsonFactory(), credential);
    endpoint = bld.setRootUrl(DEV_SERVER).setGoogleClientRequestInitializer(
        new GoogleClientRequestInitializer() {
            @Override
            public void initialize(AbstractGoogleClientRequest request)
                throws IOException {
                request.setDisableGZipContent(true);
            }
        }
    ).build();
}
Modifications made in step #3 above are highlighted.  ACCOUNT_NAME parameter is the email of the Google account that was created the Create Google account step earlier.

Restart local dev server

  1. appengine:devserver_stop
  2. appengine:devserver

Run tests

At this point, if everything is configured correctly, we can run tests, debug tests and even debug GAE endpoints deployed on the local dev server as described in Testing and debugging GAE Endpoints in Android Studio.  The only difference from the previous post is now we are working with authenticated endpoints instead of un-authenticated endpoints.
If we set incorrect account name when calling credential.setSelectedAccountName() in TestEndpointTest.setUp() method, we will get the following error:
com.google.android.gms.auth.GoogleAuthException: BadUsername
If the account name is correct, the tests should run without problems.

Friday, December 27, 2013

From an emulator to a real device

In Testing and debugging GAE Endpoints in Android Studio we used an Android emulator to test a debug GAE Endpoints running on local dev server.  Here we describe the steps required to move the client from an emulator to a real Android device.

Bind local dev server to all interfaces

By default, local dev server is listening to incoming connections only on the loopback network device. We need to change the server's configuration so it can accept incoming connections on all network interfaces.
  1. Open Project-AppEngine/pom.xml
  2. Locate <groupId>com.google.appengine</groupId> plugin
  3. Under <configuration/> tag add <address>0.0.0.0</address>
  4. Optional, add <port>8080</port> (or other value)
  5. Restart local dev server.  It is better to do this in two steps:
    • appengine:devserver_stop
    • appengine:devserver
Open a web browser on the same computer where the local dev server is running.  Both URIs should connect to the local dev server:
  • http://localhost:8080
  • http://ip-address:8080
where ip-address is the IP address of the computer running local dev server.  You should see the content of Project-AppEngine/src/main/webapp/index.html page.

Connect Android device to WiFi

Both an Android device that will be running instrumentation tests and a computer running local dev server should be connected to the same network.  After connecting an Android device to WiFi, open a web browser on the device. Type http://ip-address:8080, where ip-address is the IP address of the computer running local dev server.  If you don't see the content of Project-AppEngine/src/main/webapp/index.html page, go to the next step, otherwise skip it.

Configure firewall

The most likely reason you do not see the content of Project-AppEngine/src/main/webapp/index.html page from an Android device (or another computer) is because your computer's firewall is blocking incoming connections to the local dev server.  You need to whitelist Java so that the clients from the local network can connect to the dev server.  The steps below show how to do this on OS X 10.9.1.  Other OSes should have similar configuration options.
  1. Open System Preferences
  2. Open Security & Privacy
  3. Click Firewall tab
  4. Unlock to enable Firewall Options
  5. Go to Firewall Options
  6. Make sure that "Allow incoming connections" is enabled for Java
  7. Just in case, restart the local dev server
    • appengine:devserver_stop
    • appengine:devserver
Open a web browser on an Android device connected to local WiFi.  Type http://ip-address:8080, where ip-address is the IP address of the computer running local dev server.  After whitelisting Java, you should see the content of Project-AppEngine/src/main/webapp/index.html page.

Modify dev server URI

In Testing and debugging GAE Endpoints in Android Studio we used TestEndpointTest to run instrumentation tests. Modify TestEndpointTest.DEV_SERVER:
private static final String DEV_SERVER = "http://ip-address:8080/_ah/api/";
where ip-address is the IP address of the computer running local dev server.  You can add ip-address to the list of manually assigned IP addresses by your DHCP server.  Or, if you run a local DNS server, you can replace ip-address with the host name.

Run instrumentation tests

Connect your device to your computer through a USB cable and run an instrumentation tests as described in Testing and debugging GAE Endpoints in Android Studio.   If everything is configured correctly, the tests should run and you should be able to debug GAE endpoints in Android Studio.  Running instrumentation tests on an Android emulator should also work with this configuration.

Tuesday, December 24, 2013

Should AndroidManifest.xml for instrumentTest be customizable?

I've been trying to test authenticated GAE endpoints deployed on local Dev Server.  Authenticated endpoints require dependency on Google Play services.  I added it to my build.gradle:
dependencies

{
    // ...
    instrumentTestCompile 'com.google.android.gms:play-services:4.0.30'
    // ...
}

Now, the following error occurs when running instrumentation tests:

java.lang.IllegalStateException: A required meta-data tag in your app's AndroidManifest.xml does not exist.
You must have the following declaration within the <application/> element:
  <meta-data android:name="com.google.android.gms.version"
    android:value="@integer/google_play_services_version"/>

There seems to be no way to add the required <meta-data/> element to AndoidManifest.xml file in instrumentTest because the file is not customizable.  Just read this documentation:

The [src/instrumentTest] sourceSet should not contain an AndroidManifest.xml as it is automatically generated.

This is a big problem.  Instrumentation tests can have metadata, permissions, activities, etc, that are different from the application.  In fact, there is a Stackoverflow post that asks exactly this.

It is really surprising that this has not been brought yet to attention of the development team that works on Gradle build system for Android Studio.  This is a deal breaker.  I hope there is a workaround that I am simply missing...

Update

After adding a dependency on com.google.android.gms:play-services:4.0.30 to build.gradle, I started getting IllegalStateException described earlier.   Since Project-endpoints/src/instrumentTest/AndroidManifest.xml is not customizable, I added the required <meta-data/> tag to Project-endpoints/src/main/AndroidManifest.xml.  When I re-ran the test, I got the same error.
Fortunately, the error was easy to resolve: all that was required is to clean the project.
  • From command line: gradlew clean
  • From Android Studio: Build -> Clean Project or Build -> Rebuild Project
When gradle build system generates AndroidManifest.xml for Project-endpoints/src/instrumentTest, it takes into account Project-endpoints/src/main/AndroidManifest.xml.  All that is required is to clean the project before re-running the tests.  I haven't seen any documentation on what exactly goes from the main/AndroidManifest.xml to instrumentTest/AndroidManifest.xml, but at least <meta-data/> tags seem to be picked up.

Sunday, December 22, 2013

Testing and debugging GAE Endpoints in Android Studio

This material describes how to test GAE Endpoints code on a local Dev Server in Android Studio.  It expands on the earlier Minimalistic GAE Endpoints in Android Studio post.

Local Dev Server

Starting local Dev Server from Android Studio is easy.  
  • From Maven Projects execute
    • Project-AppEngine/Plugins/appengine/appengine:devserver
Go to http://localhost:8080/ to verify that the project is running.  The browser should display your Project-AppEngine/src/main/webapp/index.html page.

Android Emulator

By default, the local Dev Server refuses all connections except those made through http://localhost:8080/.  Since Android emulator uses a special IP address (10.0.2.2) to connect to your localhost, the easiest way to test GAE endpoints is from the emulator.
Create an Android emulator, or use an existing one.  If you create a new emulator, use Intel Atom (x86) image for better performance.
Start the emulator.  Open a browser tab and connect it to http://10.0.2.2:8080.  If the local Dev Server is running, the browser should display your Project-AppEngine/src/main/webapp/index.html page. 

Test Project Dependencies

Add google-http-client-gson dependency to Project-endpoints/build.gradle
dependencies {
    // ...
    instrumentTestCompile 'com.google.http-client:google-http-client-gson:1.17.0-rc'
}

Test Class

Create TestEndpointTest.java under Project-endpoints/src/instrumentTest/java/com/example/package
public class TestEndpointTest extends AndroidTestCase {
    private static final String DEV_SERVER = "http://10.0.2.2:8080/_ah/api/";
    private static final String TAG = TestEndpointTest.class.getSimpleName();
    private Testendpoint endpoint;

    @Override
    protected void setUp() throws Exception {
        super.setUp();
        final Testendpoint.Builder bld = new Testendpoint.Builder(
            AndroidHttp.newCompatibleTransport(), new GsonFactory(), null);
        endpoint = bld.setRootUrl(DEV_SERVER).setGoogleClientRequestInitializer(
            new GoogleClientRequestInitializer() {
                @Override
                public void initialize(AbstractGoogleClientRequest request)
                    throws IOException {
                    request.setDisableGZipContent(true);
                }
            }
        ).build();
    }

    @Override
    protected void tearDown() throws Exception {
        endpoint = null;
        super.tearDown();
    }

    public void testListEntities() throws Exception {
        final CollectionResponseTest list = endpoint.listTest().setLimit(10).execute();
        if (list.getItems() != null) {
            for (final Test t : list.getItems()) {
                Log.d(TAG, "id: " + t.getId() + ", message: " + t.getMessage());
            }
        }
    }
}
TestEndpointTest class initializes the endpoint in its setUp() method. It also has a single test method, testListEntities().  Other tests can be created similarly.

Running Tests

Place the mouse cursor on the body of testListEntities(), right-click and select 
  • Run 'testListEntities()' to run the test
  • Debug 'testListEntities()' to debug the test
When you chose Debug option, you will be debugging the client code running on Android Emulator.  Debugging server code running on the local Dev Server is explained next.

Debugging GAE Endpoints on local Dev Server

Create new Remote Run/Debug Configuration

  1. Open the dialog from Run -> Edit Configurations...
  2. Click + button at the top left corner of the dialog
  3. Select Remote from the drop-down menu
    • This will create a new debug configuration
  4. Name the configuration properly, say DevServer
  5. Copy Command line arguments for running remote JVM
    • the arguments will be used in the next step

Modify pom.xml

  1. Open Project-AppEngine/pom.xml
  2. Locate <groupId>com.google.appengine</groupId> plugin
  3. Under <configuration/> tag add <jvmFlags/> tag.
  4. Under <jvmFalgs/> tag add <jvmFalg/> and paste the command line arguments for running remote JVM. 

The resulting <configuration/> should look something like this:
<configuration>
    <enablejarclasses>false</enablejarclasses>
    <jvmflags>
        <jvmflag>-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005</jvmflag>
    </jvmflags>
</configuration>
where the contents of the <jvmFlag/> tag comes from the step #5 under Create new Remote Run/Debug Configuration.

Debugging the endpoints

  1. Restart local Dev Server
    • From Maven Projects  execute Project-AppEngine/Plugins/appengine/appengine:devserver
  2. Attach debugger to the Dev Server process
    • Select DevServer under available configurations drop-down menu
    • Click on the Debug 'DevServer' button located on the right hand side of the available configurations drop-down
    • The debugging console should print something like:
      • Connected to the target VM, address: 'localhost:5005', transport: 'socket'
  3. Under Project-AppEngine project, set breakpoint in TestEndpoint.listTest() method
  4. Under Project-endpoints project, run TestEndpointTest.testListEntities() method shown earlier.
  5. At this point, your breakpoint should be hit.

Saturday, December 21, 2013

Minimalistic GAE Endpoints in Android Studio

The goal

The goal is to remove deprecated code that is auto-generated by Android Studio.  I only want my entities in the project, no depreciated dependencies.  I will add push notifications through Google Cloud Messaging library.

Q/A

Q: Which code is deprecated?
A: Any generated code that depends on com.google.android.gcm is deprecated.

Generating initial backend and client libraries

  1. Open an existing Android project
    • Let's call it Project
    • Let's assume that the package name is com.example.package
  2. Generate App Engine Backend
    • Tools -> Google Cloud Tools -> Generated App Engine Backend
    • This will create two projects:
      • Project-AppEngine
      • Project-endpoints

Cleanup

  1. Delete
    • Under Project-AppEngine
      • google_generated
      • target
    • Under Project-AppEngine/src/main/java/com.example.package
      • Device*.java
      • Message*.java
    • From Project-AppEngine/src/main/webapp/index.html
      • scripts
      • content of the <body/> tag
      • add something like <h1>Sample Project</h1> under <body/>
    • Under Project-endpoints/src/endpoint-src/java
      • com.example.package
    • Under Project-endpoints/src/main
      • com.example.package
    • From Project-endpoints/src/main/AndroidManifest.xml
      • GCMIntentService
      • GCMBroadcastReceiver
      • RegisterActivity
      • More stuff can be removed from this file, like unused permissions.
    • From Project-endpoints/build.gradle
      • compile files('libs/gcm.jar')
      • more dependencies can be removed from this file
    • Under Project-endpoints/libs
      • gcm.jar

Adding Entities and deploying

  1. Add a test entity under Project-AppEngine/src/main/java/com.example.package
    • Test.java
  2. Generate TestEndpoint.java
    • Select Test.java
    • Tools -> Google Cloud Tools -> Generate Endpoint
  3. Deploy the project to GAE 
    • Maven Projects -> Project-AppEngine/Plugins/appengine/appengine:update
  4. At this point you can interact with TestEndpoint in APIs Explorer

Client libraries

  1. Generate client libraries
    • Select Project-AppEngine
    • Tools -> Google Cloud Tools ->  Generate Client Libraries
  2. Synchronize the project (refresh)
    • This step creates
      • Project-AppEngine/google_generated
      • Project-AppEngine/target/Project-
        AppEngine-1.0
    • Nothing is yet created under Project-endpoints/src/endpoint-src/java
  3. Generate client libraries - AGAIN!
    • Select Project-AppEngine
    • Tools -> Google Cloud Tools ->  Generate Client Libraries
  4. Synchronize the project (refresh) - AGAIN!
  5. Look under Project-endpoints/src/endpoint-src/java/com.example.project.testendpoing
    • On the second attempt the client code for TestEndpoint is generated
    • Now this code can be used in the original Android Project to communicate with GAE.

Notes

When you add a new entity under Project-AppEngine/src/main/java/com.example.package, the step of generating client libraries has to be performed twice, at least at the moment.
  • First run will modify Project-AppEngine
  • Second run will generate client code under Project-endpoints
Hopefully this will be fixed in the future.