Hi, this is YOSHIMURA, researching a wearable device at ATL. Just from an idea, I posted 3 ~ 4 articles about Google Glass, which by accident I had had exposure to Google Glass. I find the device interesting a lot. A mobile phone has the UX such as people getting a notification→starting up an application by grabbing a phone from your pocket→checking the information→finished. On the other hand, Google glass has totally a different UX from the ones on a mobile phone. For example, we can control the device by voice commands, and when we get a notification, just by nodding, we can check it with a screen indistinctly projected in the upper right corner in our sight, and when we finish it, the screen goes off by itself. The totally new UX should be needed in the applications too. I have written about those for a couple of times though.
I was faced to the difficulties of the cost and of the ways to get a real device that any formal emulator even if I want to make applications on the device. While thinking it over and over again, I finally got an idea to make a prototype of Glassware using a Nexus 5, about which I will write this time. Please note that it is possible for my story to go off the mark.
I just want you to know that this is absolutely just one of the discoveries I got through my research.
The ways I am going to discuss these are not the best and I have been in the situation those cases sometimes.
Nexus 5?
The reason why I use a Nexus 5 is that it works in the original Android 4.4 (the version as a base of XE16 or later) and is comparatively easy to use. The “original” version becomes a very important factor because I am going to make a prototype working with a different system even on an Android basis. On the other hand, because Google Glass is, if anything, similar to the Nexus S in terms of hardware specifications, we may have some cases where we had better to work with CyanogenMod 11 with the Nexus S.
Workflow
In general, the step-by-step approach is somewhat effective; first, we put an appropriate emulation layer in between, start modeling mainly Activity, and once we finish it, port it to glass.
Modeling
We insert the layer which mocks up Glass organic API like below.
LiveCard→Ongoing Notification
Though it is popular in Google ware to match Service and LiveCard together, I think it is better to emulate Service ＋Service＋Notification or Widget＋RemoteViews.
You can see those in below;
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
private LiveCard mLiveCard; @Override public void onCreate() { super.onCreate(); mLiveCard = new TimerCard(this, TAG); mLiveCard.attach(this); mLiveCard.setAction(PendingIntent.getActivity(this, 1, new Intent(this, TimerMenuActivity.class), 0)); mLiveCard.publish(LiveCard.PublishMode.REVEAL); } @Override public void onDestroy() { mLiveCard.unpublish(); mLiveCard = null; super.onDestroy(); } |
Emulation layer which gets this Android Notification activated goes as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
package com.google.android.glass; ... // Emulating LiveCard with On-going Notification public class LiveCard { final Context mContext; final String mTag; final Notification.Builder mBuilder; public LiveCard(final Content context, final String tag) { mContext = context; mTag = tag; mBuilder = new Notification.Builder(context); } public void attach(final Service service) { } public void setAction(final PendingIntent intent) { mNotificationBuilder.setContentIntent(PendingIntent.getActivity(this, 1, new Intent(this, TimerMenuActivity.class), 0)) } public void publish() { final NotificationManager nm = (NotificationManager)mContext.getSystemService(Context.NOTIFICATION_SERVICE); mNotificationBuilder .setSmallIcon(R.icon.notify) .setContentTitle("Running") .setContentText("Service is running"); nm.notify(mNotificationBuilder.build(), 1); } public void unpublish() { nm.cancel(1); } } |
Mirror API→GCM+Notification?
In the same way, we want to use Google Now for Mirror API but unfortunately for now we cannot make Google Now provide an original card. As for this, we have to set up a service that we can use Notifications with a remote control by using Google Cloud Messaging (GCM), as there is no other way to make emulation by using it.
We can write these services by using Go and Google App Engine. (note: the latest Google Cloud SDK has been changed a little bit.)
Start up with app.yaml.
|
1 2 3 4 5 6 7 8 |
application: your-application-id-goes-here version: 1 runtime: go api_version: go1 handlers: - url: /(|device)(/.*)? script: _go_app |
It is fortunate enough that there is a library called gcm which handles the GCM relationship in Go, and I use it here. All we have to do is just to install it through goapp to make it work in an development environment, but since we forget to include it when we apply it to Google App Engine with the situation as it is, make a copy to a working directory.
|
1 2 |
$ goapp get github.com/alexjlockwood/gcm $ tar -C $(goapp env | grep GOPATH | sed -e 's/GOPATH=//;s/"//g')/src/ -c github.com/alexjlockwood/gcm | tar -x |
I write how to implement the logic as reflector/reflector.go as follows; The processing of GCM relevance is made in a library, so here it is just very simple to receive the registration of a device, to write in Datastore, and to pass on a parameter which we want to spread to a library.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
package reflector import ( "fmt" "net/http" "appengine" "appengine/urlfetch" "appengine/datastore" "github.com/alexjlockwood/gcm" ) type Device struct { ID string } func init() { http.HandleFunc("/", handleNewSession) http.HandleFunc("/device", handleNewDevice) } func handleNewDevice(w http.ResponseWriter, r *http.Request) { if r.Method == "POST" { device := new(Device) device.ID = r.FormValue("regid") c := appengine.NewContext(r) if _, e := datastore.Put(c, datastore.NewKey(c, "Device", device.ID, 0, nil), device); e == nil { fmt.Fprintln(w, "{success:true}") } else { panic(e) } } else { http.NotFound(w, r) } } func handleNewSession(w http.ResponseWriter, r *http.Request) { if r.Method == "POST" { data := map[string]interface{}{"message":r.FormValue("message"))} c := appengine.NewContext(r) client := urlfetch.Client(c) sender := &gcm.Sender{ApiKey: "YOUR_APIKEY_GOES_HERE", Http: client} var devices []Device if _, e := datastore.NewQuery("Device").GetAll(c, &devices); e == nil { msg := gcm.NewMessage(data) msg.RegistrationIDs = Device_extractIDs(devices) _, e := sender.Send(msg, 3) if e == nil { fmt.Fprintln(w, "success") } else { panic(e) } } else { panic(e) } } else { http.NotFound(w, r) } } func Device_extractIDs(devices []Device) []string { var regids []string for _, t := range devices { regids = append(regids, t.ID) } return regids } |
Put them together in this way below and develop it in Google App Engine.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
$ find . ... ./app.yaml ./github.com ./github.com/alexjlockwood ./github.com/alexjlockwood/gcm ./github.com/alexjlockwood/gcm/LICENSE.md ./github.com/alexjlockwood/gcm/message.go ./github.com/alexjlockwood/gcm/README.md ./github.com/alexjlockwood/gcm/response.go ./github.com/alexjlockwood/gcm/sender.go ./github.com/alexjlockwood/gcm/sender_test.go ./reflector ./reflector/reflector.go $ appcfg.py update . ... 12:01 PM Application: your-application-id-goes-here; version: 1 12:01 PM Host: appengine.google.com 12:01 PM Starting update of app: your-application-id-goes-here, version: 1 12:01 PM Getting current resource limits. 12:01 PM Scanning files on local disk. 12:01 PM Cloning 8 application files. 12:01 PM Compilation starting. 12:01 PM Compilation: 5 files left. 12:01 PM Compilation completed. 12:01 PM Starting deployment. 12:01 PM Checking if deployment succeeded. 12:01 PM Deployment successful. 12:01 PM Checking if updated app version is serving. 12:01 PM Completed update of app: your-application-id-goes-here, version: 1 |
It’s all done with server settings. This can make us register the Android device by posting as ragid= … to http://your-application-id-goes-here.appspot.com/device and then by posting the data message=… to http://your-application-id-goes-here.appspot.com/, we can push notifications from outside.
By receiving it in a device and sending out a notification, when we use Mirror API to send out a non-obstructive notification from outside, I think that we can make emulation roughly though.
In this example, since we do not limit the target of notifications, when we post a notification and all devices receive it. It will be no problem as this is for a test only and is used in a small group.
GestureDetector→Emulate
Next, we need to prepare GestureDetector special to GDK in case of Glassware to detect gesture, while there is a GesutureDetector in the Android SDK. Let’s say there is the following code. The emulation layer to get the code working is as follows:
Glass:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
public class MainActivity extends Activity { private GestureDetector mDetector; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); mDetector = createGestureDetector(); } @Override public boolean onGenericMotionEvent(MotionEvent event) { if (mDetector != null) { return mDetector.onMotionEvent(event); } return false; } private GestureDetector createGestureDetector() { final GestureDetector ret = new GestureDetector(this); ret.setBaseListener(new GestureDetector.BaseListener() { @Override public boolean onGesture(Gesture gesture) { final Context c = MainActivity.this; if (c != null) { if (gesture == Gesture.TAP) { ... return true; } } return false; } }); return ret; } } |
I skip the implementation in the virtual logic of the detector, but this level of implementation quality is the same as the one in detection in the Android SDK normally.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
package com.google.android.glass.touchpad; ... public class GestureDetector { private Context mContext; private boolean mAlwaysConsumeEvents = false; private BaseListener mBaseListener; private FingerListener mFingerListener; private ScrollListener mScrollListener; private TwoFingerScrollListener mTwoFingerScrollListener; public GestureDetector(Context context) { mContext = context; } public static boolean isForward(Gesture gesture) { switch (gesture) { case SWIPE_RIGHT: case TWO_SWIPE_RIGHT: return true; case SWIPE_DOWN: case SWIPE_LEFT: case SWIPE_UP: case TWO_SWIPE_DOWN: case TWO_SWIPE_LEFT: case TWO_SWIPE_UP: return false; default: throw new IllegalStateException("non-slidal event"); } } public static boolean isForward(float deltaX) { return deltaX > 0.0f; } public boolean onMotionEvent(MotionEvent event) { // TBD: actual detection return mAlwaysConsumeEvents; } public GestureDetector setAlwaysConsumeEvents(boolean enabled) { mAlwaysConsumeEvents = enabled; return this; } public GestureDetector setBaseListener(GestureDetector.BaseListener listener) { mBaseListener = listener; return this; } public GestureDetector setFingerListener(GestureDetector.FingerListener listener) { mFingerListener = listener; return this; } public GestureDetector setScrollListener(GestureDetector.ScrollListener listener) { mScrollListener = listener; return this; } public GestureDetector setTwoFingerScrollListener(GestureDetector.TwoFingerScrollListener listener) { mTwoFingerScrollListener = listener; return this; } public static interface BaseListener { public boolean onGesture(Gesture gesture); } public static interface FingerListener { public boolean onFingerCountChanged(int previousCount, int currentCount); } public static interface ScrollListener { public boolean onScroll(float displacement, float delta, float velocity); } public static interface TwoFingerScrollListener { public boolean onTwoFingerScroll(float displacement, float delta, float velocity); } } |
The Gesture class in GDK is just an enum, and we need to define it since it does not have incompatibility with the Android SDK as might be expected.
|
1 2 3 4 5 |
package com.google.android.glass.touchpad; public enum Gesture { LONG_PRESS, SWIPE_DOWN, SWIPE_LEFT, SWIPE_RIGHT, SWIPE_UP, TAP, THREE_LONG_PRESS, THREE_TAP, TWO_LONG_PRESS, TWO_SWIPE_DOWN, TWO_SWIPE_LEFT, TWO_SWIPE_RIGHT, TWO_SWIPE_UP, TWO_TAP } |
At last, we need to analyze all MotionEvents in the Activity layer.
|
1 2 3 4 5 6 7 8 9 |
public class MainActivity extends Activity { ... @Override public boolean dispatchTouchEvent(MotionEvent ev) { mDetector.onMotionEvent(ev); return super.dispatchTouchEvent(ev); } ... } |
All of those steps have the code activate for Google Glass.
GCM→Standalone GCM
As Glass does not have Google Play Services, we use the one in the Standalone edition in case of using GCM. We can use the Glass in Standalone edition without any trouble, so using it makes Glass active.
Card→LayoutInflater
The Card plays a helper role in making the Glass standard layout through receiving parameters, so it is adequate to load the rough custom layout with LayoutInflater easily.
Glass:
|
1 2 3 4 |
final Card c = new Card(this); c.setText("This is card."); c.setFootnote("Can you see this?"); final View v = c.getView(); |
In case of these codes as below, we should make emulation in Android. In this example, while we do not define the method of AddImage and so on, we need to add more if needed in more complicated case.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
package com.google.android.glass.app; ... public class Card { private Context mContext; private String mText; private String mFootnote; public Card(final Context c) { mContext = c; } public CharSequence getFootnote(){ return mFootnote; } public Card setFootnote(final String footnote){ mFootnote = footnote; } public CharSequence getText(){ return mText; } public Card setText(final String text){ mText = text; } public View getView(){ final LayoutInflater inflater = (LayoutInflater)mContext.getSystemService(Context.LAYOUT_INFLATER_SERVICE); final View v = inflater.inflate(R.layout.card_text_with_footnote, null, false); ((TextView)v.findViewById(R.id.text)).setText(mText); ((TextView)v.findViewById(R.id.footnote)).setFootnote(mFootnote); return v; } } |
Menu
It is the best way to rely on system base emulation environment by clear Active＋Option Menu, instead of struggling custom designs. The appearance seems totally different when you activate Android in this way.
Glass:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
public class MainActivity extends Activity { private GestureDetector mDetector; ... @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); mDetector = createGestureDetector(); } ... @Override public boolean onGenericMotionEvent(MotionEvent event) { if (mDetector != null) { return mDetector.onMotionEvent(event); } return false; } private GestureDetector createGestureDetector() { final GestureDetector ret = new GestureDetector(this); ret.setBaseListener(new GestureDetector.BaseListener() { @Override public boolean onGesture(Gesture gesture) { final Context c = MainActivity.this; if (c != null) { if (gesture == Gesture.TAP) { final Intent intent = new Intent(this, MainMenuActivity.class); intent.setFlags(Intent.FLAG_ACTIVITY_SINGLE_TOP | Intent.FLAG_ACTIVITY_NO_HISTORY); startActivity(intent); return true; } } return false; } }); return ret; } } |
The clear Activity to launch menu is as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
// Transparent activity to launch option menus public class MainMenuActivity extends Activity { @Override public void onAttachedToWindow() { super.onAttachedToWindow(); openOptionsMenu(); } @Override public boolean onCreateOptionsMenu(final Menu menu) { getMenuInflater().inflate(R.menu.main, menu); return true; } @Override public boolean onOptionsItemSelected(final MenuItem item) { switch (item.getItemId()) { case R.id.exit_menu_item: ... } return true; } @Override public void onOptionsMenuClosed(final Menu menu) { finish(); } } |
Voice Trigger relevance
While the Meta data is ok as it is, we need to make a class reference be a stub as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
package com.google.android.glass.app; ... public class VoiceTriggers { public static final String ACTION_VOICE_TRIGGER = "com.google.android.glass.action.VOICE_TRIGGER"; public static final String EXTRA_INPUT_SPEECH = "input_speech"; public static enum Commands { ADD_AN_EVENT, CALCULATE, CALL_ME_A_CAR, CAPTURE_A_PANORAMA, CHECK_ME_IN, CHECK_THIS_OUT, CONTROL_MY_CAR, CONTROL_MY_HOME, CREATE_A_3D_MODEL, EXPLORE_NEARBY, EXPLORE_THE_STARS, FIND_A_BIKE, FIND_A_DENTIST, FIND_A_DOCTOR, FIND_A_FLIGHT, FIND_A_HOSPITAL, FIND_A_PASSAGE, FIND_A_PLACE, FIND_A_PLACE_TO_STAY, FIND_A_PRODUCT, FIND_A_RECIPE, FIND_A_VIDEO, FIND_A_WEBSITE, FIND_REVIEWS, FIND_THE_EXCHANGE_RATE, FIND_THE_PRICE, FLIP_A_COIN, GIVE_ME_FEEDBACK, HELP_ME_RELAX, HELP_ME_SIGN_IN, KEEP_ME_AWAKE, LEARN_AN_INSTRUMENT, LEARN_A_SONG, LISTEN_TO, LOCATE_A_SATELLITE, LOG_A_MEAL, LOOK_UP_THE_DEFINITION, MAGNIFY_THIS, MAKE_A_REQUEST, MAKE_A_RESERVATION, PICK_A_CARD, PLAY_A_GAME, POST_AN_UPDATE, POST_A_QUESTION, RECOGNIZE_THIS, RECOGNIZE_THIS_SONG, RECORD_A_RECIPE, RECORD_A_VIDEO, REMEMBER_THIS, REMEMBER_WHERE_I_AM, REMIND_ME, ROLL_THE_DICE, SCAN_A_PRODUCT, SEND_MONEY, SHARE_MY_LOCATION, SHARE_THIS_WITH, SHOW_A_COMPASS, SHOW_ME_ANALYTICS, SHOW_ME_A_DEMO, SHOW_ME_MY_ACCOUNT, SHOW_ME_MY_SPEED, SHOW_ME_THE_NEWS, SHOW_ME_THE_WEATHER, SHOW_ME_TRANSIT_TIMES, SHOW_MY_PICTURES, SHOW_MY_VIDEOS, SHOW_SONG_LYRICS, START_A_BIKE_RIDE, START_A_FLIGHT, START_A_ROUND_OF_GOLF, START_A_RUN, START_A_STOPWATCH, START_A_TIMER, START_A_WORKOUT, START_BROADCASTING, START_COACHING, START_IMAGING, START_PRESENTING, TAKE_A_NOTE, TAKE_A_PICTURE, TEACH_ME_ABOUT, TRANSLATE_THIS, TUNE_AN_INSTRUMENT, TURN_THE_FLASHLIGHT_ON, WATCH_MY_SWING } } |
Since just with this procedure it does not listen to the voice command, we need to be prepared to activate the Service supported by voice commands separately from android.intent.action.MAIN/android.intent.category.LAUNCHER
※Please refer to the article in lifehacker for more detailed procedures.
Porting
We need to fix it if there is an incompatibility by taking out an emulation layer. If the application itself is simple and the layer design goes well, there will be not such a big problem. But I think the following can be a problem in porting in a real situation.
OpenGL ES 2.0
GAs the GPU installed on Glass is a SGX540 which is the same as the Nexux S, it is totally different from the Nexus5. For this reason, it often fails in compiling shaders. In this case, make sure you check the activation with Nexus S＋CyanogenMod 11.
Generation of heat
While the CPU itself in Glass is the same as in the Nexus S, the form factor is so different that it generates heat immediately with heavy use. Be aware of the message “ok, glass” and “Glass must cool down to run smoothly.” if it temperature gets high.
Summary
Google Glass and Android mobile devices have a lots in common. Can I share the image that the more you go away from the front end, the more both devices have points in common? The differences I mentioned above are what I found by accident during participating in my research, and this is most likely about it. As I worried before, this entry went off the mark. But I hope this will be helpful for developing efficiency improvement with developing for Google Glass.