KorhalResearch
Research
- Figuring out stuff for the lolz.
- ??
- ??
- Profit!
UI
Ok , we've got qml2 kicking on Surfaceflinger at 60FPS. YouTube Video
Input comes from evdev for now, until we figure out how to reuse androids ui management.
compositing multiple windows and stuff isn't clear yet either.
Services
Android is designed to expose services through the java public API. Since we're running native, reusing services is a mixed bag. We need to find a process border that:
- Keeps Qt and Android seperate things from a legal perspective.
- Allows us to rip out services and replace them with other stuff.
- Doesn't change too frequently upstream.
Let's see what we can do with just using binder directly. II'll just show the exciting parts. You can read up all the interfaces form the aidl.
Since google likes to mess with method index ordering, (for example commit 25101b0b9a84571ead15b26e9f4cd9c4298d7823 in frameworks/base.) it's pretty clear that we need to parse the aidls to get the correct order instead of hardcoding them. The numbers shown here are for 4.0.1
Telephony
frameworks/base/telephony/java/com/android/internal/telephony/ITelephony.aidl
| Method Index | prototype | effect |
|---|---|---|
| 0 | void dial(String number); | opens the dialer ui with the given number |
| 1 | void call(String number); | calls the given number |
| 3 | endCall() | guess what |
| 4 | answerRingingCall() | Answer the phone when it rings |
| 5 | silenceRinger() | Or don't |
| ? | boolean setRadio(boolean turnOn); | switch on/off radio (flight mode) |
We apparantly can get a notification for incomming calls from ITelephonyRegistry.aidl. cool, but what if we want our own call ui? Well, darn, those service interfaces are part of the ui. So replacing the UI involves talking to RIL directly.
Connectivity Management
./frameworks/base/core/java/android/net/IConnectivityManager.aidl
| Method Index | prototype | *effect* |
| 2 | NetworkInfo getActiveNetworkInfo(); | i get nothing |
| 12 | boolean setRadios(boolean onOff); | disables/enables all internet connections but leaves the actual radio emission on (not flight mode) |
| 25 | String[] getTetheredIfaces | nothing |
App Integration
Headless
am start -a android.intent.action.MAIN -n com.android.browser/.BrowserActivity
am start -a android.intent.action.MAIN -n com.android.settings/.Settings
wohoo magic. now let's try headless. Looks like this was intended for Google Q, which has no UI at all.
setprop ro.config.headless 1
stop zygote
start zygote
am start -a android.intent.action.MAIN -n com.android.browser/.BrowserActivity
E/ActivityManager( 1733): Starting activities not supported on headless device: ActivityRecord{478c83e8 com.android.browser/.BrowserActivity}
oh well, worth a try. We'll start systemserver again and see how much we can remove without breaking apps.
| Service | started from | effect of removal |
|---|---|---|
| SystemUi | SystemServer.java | missing softbuttons and statusbar |
| WindowManager | SystemServer.java | Crashes. Systemserver has refs all over the place. |
| ActivityManager | SystemServer.java | Ah well, what did you expect… |
| Launcher | am/ActivityManagerService.java:2122 | Input stops working. can't get past lockscreen |
SystemUi is an app (frameworks/base/packages/SystemUI/src/com/android/systemui), same for homescreen (./packages/apps/Launcher2/) but that's about it. Everything else is systemserver, period.
Application startup path
Android uses a Async technique to start applications. The Zygote process preloads all needed libraries and just gets forked for every new started app, the client process then loads the apk and start activities or services.
1. Launcher
/frameworks/base/core/java/android/app/
First a user clicks/touches a shortcut on the android launcher, launcher calls Activity::startActivity() which forwards the call to ActivityManagerNative::startActivity()
int result = ActivityManagerNative.getDefault()
.startActivity(whoThread, intent,
intent.resolveTypeIfNeeded(who.getContentResolver()),
token, target != null ? target.mEmbeddedID : null,
requestCode, 0, null, null, options);
ActivityManagerNative now executes a Binder transaction to the am Server.
2. ActivityManager
/frameworks/base/services/java/com/android/server/am
ActivityManagerNative now receives the binder call and executes the real startActivity implementation: http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#2349
which calls :
1. ActivityStack::startActivityMayWait
2. ActivityStack::startActivityUncheckedLocked
3. ActivityStack::startActivityLocked (overloaded)
4. ActivityStack::resumeTopActivityLocked
- http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1380
- I assume this function is called for existing and not existing processes, the code is hard to read here
5. ActivityStack::startSpecificActivityLocked
6. ActivityManagerService::startProcessLocked
7. ActivityManagerService::startProcessLocked (overload)
startProcessLocked will now execute the new Process:
// Start the process. It will either succeed and return a result containing
// the PID of the new process, or else throw a RuntimeException.
Process.ProcessStartResult startResult = Process.start("android.app.ActivityThread",
app.processName, uid, uid, gids, debugFlags,
app.info.targetSdkVersion, null);
Process.start tells Zygote on its socket (/dev/socket/zygote) to fork and load ActivityThread Zygote responds with the forked pid, which am puts into a list.
this.mPidsSelfLocked.put(startResult.pid, app);
From now on the currently created Process is responsible for getting back to the am server, to register itself and ask for the needed informations about the application it has to start.
3. Zygote- ActivityThread
/frameworks/base/core/java/android/app/
The newly created application enters the main function in ActivityThread: http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#4720
One of the first actions it has to do is talk back to the am-server that it has started and is now ready to load the application. This is done in the ActivityThread::attach() function: http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#attach
//get the default ActivityManager (am-server)
IActivityManager mgr = ActivityManagerNative.getDefault();
try {
//tell the am server that we are ready to load the application
mgr.attachApplication(mAppThread);
} catch (RemoteException ex) {
//Ignore
}
Control now goes back to the am-server to setup the process , mgr.attachApplication should block until the server is finished
4. ActivityManager attach
/frameworks/base/services/java/com/android/server/am
The function ActivityManagerService::attachApplicationLocked is executed
, it puts together all needed informations and calls the bindApplication func on the ActivityThread interface.
5. Zygote - load app
/frameworks/base/core/java/android/app/
The client side bindApplication function now starts to load and start the activity according to the informations it received from the am server.
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#683
the app gets loaded from apkg
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/LoadedApk.java#479
6. App - user code
finally we get onCreate in our code
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#869
7. ActivityManager start
somewhere stuff happens through messages
5. Zygote - show window
through magic messages this happens:
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#1950
which calls Activity.attach
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#4959
mWindow = PolicyManager.makeNewWindow(this);
wew that was hard to find.
Graphics stack
Information gathered from a quick skimming of the sources
native/services/surfaceflinger/DisplayHardware/DisplayHardware.cpp
- appears to allocate a FramebufferNativeWindow()
- gets an ANativeWindow * from it
- which can be sent to eglCreateWindowSurface()
- holds a reference to SurfaceFlinger
- has a HWComposer& getHWComposer() ?
- lets you register a VSyncHandler callback object
- single-process Qt should be easily doable using this approach
native/libs/ui/GraphicBufferAllocator.cpp
- allocates graphics buffers
- constructor does gralloc_open with a hw module from hw_get_module with GRALLOC_HARDWARE_MODULE_ID
- destructor does galloc_close
- gralloc_open returns a device of type alloc_device_t
- alloc_device_t has an alloc function that returns a buffer_handle_t given a width/height/usage/format/stride
- and a free function, naturally
native/libs/ui/GraphicBuffer.cpp
- has some code for serializing buffer_handle_t's (flatten / unflatten)
- locks / unlocks buffer_handle_t via GraphicBufferMapper which just forwards to the hw_get_module GRALLOC_HARDWARE_MODULE_ID, similar to GraphicsBufferAllocator
native/libs/gui/BufferQueue.cpp
- from the comments
// BufferQueue manages a pool of gralloc memory slots to be used // by producers and consumers.
- has a consumer and producer facing interface
- supports queuing / dequeuing / canceling buffers, draining the buffer queue, etc
- on the consumer side, acquiring and releasing buffers
- uses eglClientWaitSyncKHR() to synchronize
- used by both EGL, camera, and media APIs
native/libs/gui/SurfaceTexture.cpp
- has code for mapping GraphicBuffer's to textures
- interacts with the buffer queue
native/libs/gui/Surface.cpp
- client side interface to SurfaceTexture
- SurfaceControl controls how a surface is positioned, cropped, etc
- Surface provides the EGLNativeWindowType
native/services/surfaceflinger/Layer.cpp
- used by SurfaceFlinger to actually render Surface's (happens in Layer::onDraw)
- onDraw is called from base class LayerBase, LayerBase::draw or LayerBase::drawForSreenShot (lol)
native/services/surfaceflinger/SurfaceFlinger.cpp
- creates a DisplayHardware object
- allocates a control block (for communication with clients?)
- seems to only support one display at the time, has some comments about multiple displays
- does makeCurrent and some GL initialization
- starts an EventThread
- starts sleep management on the DisplayHardware object
- triggers the boot animation
- does dirty region handling / partial updates (unlike QML scene graph)
- composeSurfaces calls Layer::draw on each of the layers
Single-process non-composited Qt
Non-composited (i.e. not running surfaceflinger, but running Qt directly on the framebuffer with EGL/GLES2) is now possible with the following patch and its dependencies: https://codereview.qt.io/#change,38199
To get a Qt application to run at start-up, you'll want to edit system/core/rootdir/init.rc.
Comment out the surfaceflinger bits like so:
#service surfaceflinger /system/bin/surfaceflinger
# class main
# user system
# group graphics
# onrestart restart zygote
And add a service right below for the Qt application you want to run, example:
service myqtapp /system/bin/qmlscene /data/someqml.qml -plugin evdevtouch:/dev/input/event0
class main
user system
group graphics input
setenv QT_QPA_EGLFS_NO_SURFACEFLINGER 1
setenv QT_QPA_EGLFS_HIDECURSOR 1
setenv QML_FORCE_THREADED_RENDERER 1