KorhalResearch

Research

1. Figuring out stuff for the lolz.
2. ??
3. ??
4. Profit!

UI

Ok , we've got qml2 kicking on Surfaceflinger at 60FPS. YouTube Video

Input comes from evdev for now, until we figure out how to reuse androids ui management.

compositing multiple windows and stuff isn't clear yet either.

Services

Android is designed to expose services through the java public API. Since we're running native, reusing services is a mixed bag. We need to find a process border that:

• Keeps Qt and Android seperate things from a legal perspective.
• Allows us to rip out services and replace them with other stuff.
• Doesn't change too frequently upstream.

Let's see what we can do with just using binder directly. II'll just show the exciting parts. You can read up all the interfaces form the aidl.

Since google likes to mess with method index ordering, (for example commit 25101b0b9a84571ead15b26e9f4cd9c4298d7823 in frameworks/base.) it's pretty clear that we need to parse the aidls to get the correct order instead of hardcoding them. The numbers shown here are for 4.0.1

Telephony

frameworks/base/telephony/java/com/android/internal/telephony/ITelephony.aidl

Method Index	prototype	effect
0	void dial(String number);	opens the dialer ui with the given number
1	void call(String number);	calls the given number
3	endCall()	guess what
4	answerRingingCall()	Answer the phone when it rings
5	silenceRinger()	Or don't
?	boolean setRadio(boolean turnOn);	switch on/off radio (flight mode)

We apparantly can get a notification for incomming calls from ITelephonyRegistry.aidl. cool, but what if we want our own call ui? Well, darn, those service interfaces are part of the ui. So replacing the UI involves talking to RIL directly.

Connectivity Management

./frameworks/base/core/java/android/net/IConnectivityManager.aidl

Method Index	prototype	*effect*
2	NetworkInfo getActiveNetworkInfo();	i get nothing
12	boolean setRadios(boolean onOff);	disables/enables all internet connections but leaves the actual radio emission on (not flight mode)
25	String[] getTetheredIfaces	nothing

App Integration

Headless

am start -a android.intent.action.MAIN -n com.android.browser/.BrowserActivity
am start -a android.intent.action.MAIN -n com.android.settings/.Settings

wohoo magic. now let's try headless. Looks like this was intended for Google Q, which has no UI at all.

setprop ro.config.headless 1
stop zygote
start zygote
am start -a android.intent.action.MAIN -n com.android.browser/.BrowserActivity

E/ActivityManager( 1733): Starting activities not supported on headless device: ActivityRecord{478c83e8 com.android.browser/.BrowserActivity}

oh well, worth a try. We'll start systemserver again and see how much we can remove without breaking apps.

Service	started from	effect of removal
SystemUi	SystemServer.java	missing softbuttons and statusbar
WindowManager	SystemServer.java	Crashes. Systemserver has refs all over the place.
ActivityManager	SystemServer.java	Ah well, what did you expect…
Launcher	am/ActivityManagerService.java:2122	Input stops working. can't get past lockscreen

SystemUi is an app (frameworks/base/packages/SystemUI/src/com/android/systemui), same for homescreen (./packages/apps/Launcher2/) but that's about it. Everything else is systemserver, period.

Application startup path

Android uses a Async technique to start applications. The Zygote process preloads all needed libraries and just gets forked for every new started app, the client process then loads the apk and start activities or services.

1. Launcher

/frameworks/base/core/java/android/app/

First a user clicks/touches a shortcut on the android launcher, launcher calls Activity::startActivity() which forwards the call to ActivityManagerNative::startActivity()

 int result = ActivityManagerNative.getDefault()
 .startActivity(whoThread, intent,
 intent.resolveTypeIfNeeded(who.getContentResolver()),
 token, target != null ? target.mEmbeddedID : null,
 requestCode, 0, null, null, options);

ActivityManagerNative now executes a Binder transaction to the am Server.

2. ActivityManager

/frameworks/base/services/java/com/android/server/am

ActivityManagerNative now receives the binder call and executes the real startActivity implementation: http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#2349

which calls :

1. ActivityStack::startActivityMayWait

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#2998

2. ActivityStack::startActivityUncheckedLocked

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#2384

3. ActivityStack::startActivityLocked (overloaded)

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1749

4. ActivityStack::resumeTopActivityLocked

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1380

I assume this function is called for existing and not existing processes, the code is hard to read here

5. ActivityStack::startSpecificActivityLocked

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#startSpecificActivityLocked

6. ActivityManagerService::startProcessLocked

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#1883

7. ActivityManagerService::startProcessLocked (overload)

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#1987

startProcessLocked will now execute the new Process:

// Start the process. It will either succeed and return a result containing
// the PID of the new process, or else throw a RuntimeException.
Process.ProcessStartResult startResult = Process.start("android.app.ActivityThread",
 app.processName, uid, uid, gids, debugFlags,
 app.info.targetSdkVersion, null);

Process.start tells Zygote on its socket (/dev/socket/zygote) to fork and load ActivityThread Zygote responds with the forked pid, which am puts into a list.

this.mPidsSelfLocked.put(startResult.pid, app);

From now on the currently created Process is responsible for getting back to the am server, to register itself and ask for the needed informations about the application it has to start.

3. Zygote- ActivityThread

/frameworks/base/core/java/android/app/

The newly created application enters the main function in ActivityThread: http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#4720

One of the first actions it has to do is talk back to the am-server that it has started and is now ready to load the application. This is done in the ActivityThread::attach() function: http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#attach

//get the default ActivityManager (am-server)
IActivityManager mgr = ActivityManagerNative.getDefault();
try {
 //tell the am server that we are ready to load the application
 mgr.attachApplication(mAppThread);
} catch (RemoteException ex) {
 //Ignore
}

Control now goes back to the am-server to setup the process , mgr.attachApplication should block until the server is finished

4. ActivityManager attach

/frameworks/base/services/java/com/android/server/am

The function ActivityManagerService::attachApplicationLocked is executed

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#attachApplicationLocked

, it puts together all needed informations and calls the bindApplication func on the ActivityThread interface.

5. Zygote - load app

/frameworks/base/core/java/android/app/

The client side bindApplication function now starts to load and start the activity according to the informations it received from the am server.

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#683

the app gets loaded from apkg

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/LoadedApk.java#479

6. App - user code

finally we get onCreate in our code

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#869

7. ActivityManager start

somewhere stuff happens through messages

5. Zygote - show window

through magic messages this happens:

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#1950

which calls Activity.attach

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#4959

mWindow = PolicyManager.makeNewWindow(this);

wew that was hard to find.

Graphics stack

Information gathered from a quick skimming of the sources

native/services/surfaceflinger/DisplayHardware/DisplayHardware.cpp

• appears to allocate a FramebufferNativeWindow()
• gets an ANativeWindow * from it
• which can be sent to eglCreateWindowSurface()
• holds a reference to SurfaceFlinger
• has a HWComposer& getHWComposer() ?
• lets you register a VSyncHandler callback object
• single-process Qt should be easily doable using this approach

native/libs/ui/GraphicBufferAllocator.cpp

• allocates graphics buffers
• constructor does gralloc_open with a hw module from hw_get_module with GRALLOC_HARDWARE_MODULE_ID
• destructor does galloc_close
• gralloc_open returns a device of type alloc_device_t
• alloc_device_t has an alloc function that returns a buffer_handle_t given a width/height/usage/format/stride
• and a free function, naturally

native/libs/ui/GraphicBuffer.cpp

• has some code for serializing buffer_handle_t's (flatten / unflatten)
• locks / unlocks buffer_handle_t via GraphicBufferMapper which just forwards to the hw_get_module GRALLOC_HARDWARE_MODULE_ID, similar to GraphicsBufferAllocator

native/libs/gui/BufferQueue.cpp

• from the comments

// BufferQueue manages a pool of gralloc memory slots to be used
// by producers and consumers.

• has a consumer and producer facing interface
• supports queuing / dequeuing / canceling buffers, draining the buffer queue, etc
• on the consumer side, acquiring and releasing buffers
• uses eglClientWaitSyncKHR() to synchronize
• used by both EGL, camera, and media APIs

native/libs/gui/SurfaceTexture.cpp

• has code for mapping GraphicBuffer's to textures
• interacts with the buffer queue

native/libs/gui/Surface.cpp

• client side interface to SurfaceTexture
• SurfaceControl controls how a surface is positioned, cropped, etc
• Surface provides the EGLNativeWindowType

native/services/surfaceflinger/Layer.cpp

• used by SurfaceFlinger to actually render Surface's (happens in Layer::onDraw)
• onDraw is called from base class LayerBase, LayerBase::draw or LayerBase::drawForSreenShot (lol)

native/services/surfaceflinger/SurfaceFlinger.cpp

• creates a DisplayHardware object
• allocates a control block (for communication with clients?)
• seems to only support one display at the time, has some comments about multiple displays
• does makeCurrent and some GL initialization
• starts an EventThread
• starts sleep management on the DisplayHardware object
• triggers the boot animation
• does dirty region handling / partial updates (unlike QML scene graph)
• composeSurfaces calls Layer::draw on each of the layers

Single-process non-composited Qt

Non-composited (i.e. not running surfaceflinger, but running Qt directly on the framebuffer with EGL/GLES2) is now possible with the following patch and its dependencies: https://codereview.qt.io/#change,38199

To get a Qt application to run at start-up, you'll want to edit system/core/rootdir/init.rc.

Comment out the surfaceflinger bits like so:

#service surfaceflinger /system/bin/surfaceflinger
# class main
# user system
# group graphics
# onrestart restart zygote

And add a service right below for the Qt application you want to run, example:

service myqtapp /system/bin/qmlscene /data/someqml.qml -plugin evdevtouch:/dev/input/event0
 class main
 user system
 group graphics input
 setenv QT_QPA_EGLFS_NO_SURFACEFLINGER 1
 setenv QT_QPA_EGLFS_HIDECURSOR 1
 setenv QML_FORCE_THREADED_RENDERER 1