KorhalResearch: Difference between revisions

From Qt Wiki
Jump to navigation Jump to search
No edit summary
 
No edit summary
Line 1: Line 1:
=Research=
[toc align_right="yes" depth="5"]
 
= Research =


# Figuring out stuff for the lolz.
# Figuring out stuff for the lolz.
# ??
# ??
# ??
# ??
# ######!
# Profit!


==UI==
== UI ==


Ok , we’ve got qml2 kicking on Surfaceflinger at 60FPS.<br />http://www.youtube.com/watch?v=TEkubKZaUt8
Ok , we've got qml2 kicking on Surfaceflinger at 60FPS.<br />http://www.youtube.com/watch?v=TEkubKZaUt8


Input comes from evdev for now, until we figure out how to reuse androids ui management.
Input comes from evdev for now, until we figure out how to reuse androids ui management.


compositing multiple windows and stuff isn’t clear yet either.
compositing multiple windows and stuff isn't clear yet either.


==Services==
== Services ==


Android is designed to expose services through the java public <span class="caps">API</span>. Since we’re running native, reusing services is a mixed bag.<br /> We need to find a process border that:
Android is designed to expose services through the java public API. Since we're running native, reusing services is a mixed bag.<br />We need to find a process border that:


* Keeps Qt and Android seperate things from a legal perspective.
* Keeps Qt and Android seperate things from a legal perspective.
* Allows us to rip out services and replace them with other stuff.
* Allows us to rip out services and replace them with other stuff.
* Doesn’t change too frequently upstream.
* Doesn't change too frequently upstream.


Let’s see what we can do with just using binder directly.<br /> II’ll just show the exciting parts. You can read up all the interfaces form the aidl.
Let's see what we can do with just using binder directly.<br />II'll just show the exciting parts. You can read up all the interfaces form the aidl.


Since google likes to mess with method index ordering, <br /> (for example commit 25101b0b9a84571ead15b26e9f4cd9c4298d7823 in frameworks/base.)<br /> it’s pretty clear that we need to parse the aidls to get the correct order instead of hardcoding them.<br /> The numbers shown here are for 4.0.1
Since google likes to mess with method index ordering,<br />(for example commit 25101b0b9a84571ead15b26e9f4cd9c4298d7823 in frameworks/base.)<br />it's pretty clear that we need to parse the aidls to get the correct order instead of hardcoding them.<br />The numbers shown here are for 4.0.1


====Telephony====
==== Telephony ====


''frameworks/base/telephony/java/com/android/internal/telephony/ITelephony.aidl''
''frameworks/base/telephony/java/com/android/internal/telephony/ITelephony.aidl''


{| class="infotable line"
{|
| '''Method Index'''
|'''Method Index'''
| '''prototype'''
|'''prototype'''
| '''effect'''
|*effect*
|-
|-
| 0
|0
| void dial(String number);
|void dial(String number);
| opens the dialer ui with the given number
|opens the dialer ui with the given number
|-
|-
| 1
|1
| void call(String number);
|void call(String number);
| calls the given number
|calls the given number
|-
|-
| 3
|3
| endCall()
|endCall()
| guess what
|guess what
|-
|-
| 4
|4
| answerRingingCall()
|answerRingingCall()
| Answer the phone when it rings
|Answer the phone when it rings
|-
|-
| 5
|5
| silenceRinger()
|silenceRinger()
| Or don’t
|Or don't
|-
|-
| ?
|?
| boolean setRadio(boolean turnOn);
|boolean setRadio(boolean turnOn);
| switch on/off radio (flight mode)
|switch on/off radio (flight mode)
|}
|}


We apparantly can get a notification for incomming calls from ITelephonyRegistry.aidl.<br /> cool, but what if we want our own call ui?<br /> Well, darn, those service interfaces are '''part''' of the ui.<br /> So replacing the UI involves talking to <span class="caps">RIL</span> directly.
We apparantly can get a notification for incomming calls from ITelephonyRegistry.aidl.<br />cool, but what if we want our own call ui?<br />Well, darn, those service interfaces are '''part''' of the ui.<br />So replacing the UI involves talking to RIL directly.


====Connectivity Management====
==== Connectivity Management ====


''./frameworks/base/core/java/android/net/IConnectivityManager.aidl''
''./frameworks/base/core/java/android/net/IConnectivityManager.aidl''


{| class="infotable line"
{|
| '''Method Index'''
|'''Method Index'''
| '''prototype'''
|'''prototype'''
| '''effect'''
|*effect*
|-
|-
| 2
|2
| NetworkInfo getActiveNetworkInfo();
|NetworkInfo getActiveNetworkInfo();
| i get nothing
|i get nothing
|-
|-
| 12
|12
| boolean setRadios(boolean onOff);
|boolean setRadios(boolean onOff);
| disables/enables all internet connections but leaves the actual radio emission on (not flight mode)
|disables/enables all internet connections but leaves the actual radio emission on (not flight mode)
|-
|-
| 25
|25
| String[] getTetheredIfaces
|String[] getTetheredIfaces
| nothing
|nothing
|}
|}


==App Integration==
== App Integration ==
 
===Headless===
 
wohoo magic. now let’s try headless.<br /> Looks like this was intended for Google Q, which has no UI at all.


E/ActivityManager( 1733): Starting activities not supported on headless device: ActivityRecord{478c83e8 com.android.browser/.BrowserActivity}
=== Headless ===


oh well, worth a try. We’ll start systemserver again and see how much we can remove without breaking apps.
<code><br />am start -a android.intent.action.MAIN -n com.android.browser/.BrowserActivity<br />am start -a android.intent.action.MAIN -n com.android.settings/.Settings<br /></code>
 
{| class="infotable line"
| '''Service'''
| '''started from'''
| '''effect of removal'''
|-
| SystemUi
| SystemServer.java
| missing softbuttons and statusbar
|-
| WindowManager
| SystemServer.java
| Crashes. Systemserver has refs all over the place.
|-
| ActivityManager
| SystemServer.java
| Ah well, what did you expect…
|-
| Launcher
| am/ActivityManagerService.java:2122
| Input stops working. can’t get past lockscreen
|}


SystemUi is an app (frameworks/base/packages/SystemUI/src/com/android/systemui),<br /> same for homescreen (./packages/apps/Launcher2/) <br /> but that’s about it. Everything else is systemserver, period.
wohoo magic. now let's try headless.<br />Looks like this was intended for Google Q, which has no UI at all.


===Application startup path===
<code><br />setprop ro.config.headless 1<br />stop zygote<br />start zygote<br />am start -a android.intent.action.MAIN <s>n com.android.browser/.BrowserActivity<br /></code>
<br />E/ActivityManager( 1733): Starting activities not supported on headless device: ActivityRecord{478c83e8 com.android.browser/.BrowserActivity}
<br />oh well, worth a try. We'll start systemserver again and see how much we can remove without breaking apps.
<br />| '''Service''' | '''started from''' | '''effect of removal''' |<br />| SystemUi | SystemServer.java | missing softbuttons and statusbar |<br />| WindowManager | SystemServer.java | Crashes. Systemserver has refs all over the place. |<br />| ActivityManager | SystemServer.java | Ah well, what did you expect… |<br />| Launcher | am/ActivityManagerService.java:2122 | Input stops working. can't get past lockscreen |
<br />SystemUi is an app (frameworks/base/packages/SystemUI/src/com/android/systemui),<br />same for homescreen (./packages/apps/Launcher2/)<br />but that's about it. Everything else is systemserver, period.
<br />h3. Application startup path
<br />Android uses a Async technique to start applications. The Zygote process preloads all needed libraries and just<br />gets forked for every new started app, the client process then loads the apk and start activities or services.
<br />h4. 1. Launcher
<br />''/frameworks/base/core/java/android/app/''
<br />First a user clicks/touches a shortcut on the android launcher,<br />launcher calls Activity::startActivity() which forwards the call to<br />ActivityManagerNative::startActivity()
<br /><code><br /> int result = ActivityManagerNative.getDefault()<br /> .startActivity(whoThread, intent,<br /> intent.resolveTypeIfNeeded(who.getContentResolver()),<br /> token, target != null ? target.mEmbeddedID : null,<br /> requestCode, 0, null, null, options);<br /></code>
<br />ActivityManagerNative now executes a Binder transaction to the am Server.
<br />h4. 2. ActivityManager
<br />''/frameworks/base/services/java/com/android/server/am''
<br />ActivityManagerNative now receives the binder call and executes the real startActivity implementation:<br />http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#2349
<br />which calls :<br /># ActivityStack::startActivityMayWait<br /> http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#2998<br /># ActivityStack::startActivityUncheckedLocked<br /> http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#2384<br /># ActivityStack::startActivityLocked (overloaded)<br /> http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1749<br /># ActivityStack::resumeTopActivityLocked<br /> http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1380<br /> I assume this function is called for existing and not existing processes, the code is hard to read here<br /># ActivityStack::startSpecificActivityLocked<br /> http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#startSpecificActivityLocked<br /># ActivityManagerService::startProcessLocked<br /> http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#1883<br /># ActivityManagerService::startProcessLocked (overload)<br /> http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#1987


Android uses a Async technique to start applications. The Zygote process preloads all needed libraries and just <br /> gets forked for every new started app, the client process then loads the apk and start activities or services.
<br />startProcessLocked will now execute the new Process:<br /><code><br />// Start the process. It will either succeed and return a result containing<br />// the PID of the new process, or else throw a RuntimeException.<br />Process.ProcessStartResult startResult = Process.start(&quot;android.app.ActivityThread&amp;quot;,<br /> app.processName, uid, uid, gids, debugFlags,<br /> app.info.targetSdkVersion, null);<br /></code>
<br />Process.start tells Zygote on its socket (/dev/socket/zygote) to fork and load ActivityThread<br />Zygote responds with the forked pid, which am puts into a list.


====1. Launcher====
<br /><code><br />this.mPidsSelfLocked.put(startResult.pid, app);<br /></code>
<br />From now on the currently created Process is responsible for getting back to the am server,<br />to register itself and ask for the needed informations about the application it has to start.
<br />h4. 3. Zygote</s> ActivityThread


''/frameworks/base/core/java/android/app/''
''/frameworks/base/core/java/android/app/''


First a user clicks/touches a shortcut on the android launcher,<br /> launcher calls Activity::startActivity() which forwards the call to <br /> ActivityManagerNative::startActivity()
The newly created application enters the main function in ActivityThread:<br />http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#4720


ActivityManagerNative now executes a Binder transaction to the am Server.
One of the first actions it has to do is talk back to the am-server that it has started and<br />is now ready to load the application.<br />This is done in the ActivityThread::attach() function:<br />http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#attach


====2. ActivityManager====
<code><br />//get the default ActivityManager (am-server)<br />IActivityManager mgr = ActivityManagerNative.getDefault();<br />try {<br /> //tell the am server that we are ready to load the application<br /> mgr.attachApplication(mAppThread);<br />} catch (RemoteException ex) {<br /> //Ignore<br />}<br /></code>
 
''/frameworks/base/services/java/com/android/server/am''
 
ActivityManagerNative now receives the binder call and executes the real startActivity implementation:<br />http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#2349
 
which calls :
 
# ActivityStack::startActivityMayWait http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#2998
# ActivityStack::startActivityUncheckedLocked http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#2384
# ActivityStack::startActivityLocked (overloaded) http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1749
# ActivityStack::resumeTopActivityLocked http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1380 I assume this function is called for existing and not existing processes, the code is hard to read here
# ActivityStack::startSpecificActivityLocked http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#startSpecificActivityLocked
# ActivityManagerService::startProcessLocked http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#1883
# ActivityManagerService::startProcessLocked (overload) http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#1987
 
startProcessLocked will now execute the new Process:<br />
 
Process.start tells Zygote on its socket (/dev/socket/zygote) to fork and load ActivityThread <br /> Zygote responds with the forked pid, which am puts into a list.
 
From now on the currently created Process is responsible for getting back to the am server,<br /> to register itself and ask for the needed informations about the application it has to start.
 
====3. Zygote – ActivityThread====
 
''/frameworks/base/core/java/android/app/''
 
The newly created application enters the main function in ActivityThread:<br />http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#4720
 
One of the first actions it has to do is talk back to the am-server that it has started and <br /> is now ready to load the application.<br /> This is done in the ActivityThread::attach() function:<br />http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#attach


Control now goes back to the am-server to setup the process , mgr.attachApplication should block until the server is finished
Control now goes back to the am-server to setup the process , mgr.attachApplication should block until the server is finished


====4. ActivityManager attach====
==== 4. ActivityManager attach ====


''/frameworks/base/services/java/com/android/server/am''
''/frameworks/base/services/java/com/android/server/am''
Line 172: Line 138:
, it puts together all needed informations and calls the bindApplication func on the ActivityThread interface.
, it puts together all needed informations and calls the bindApplication func on the ActivityThread interface.


====5. Zygote load app====
==== 5. Zygote - load app ====


''/frameworks/base/core/java/android/app/''
''/frameworks/base/core/java/android/app/''


The client side bindApplication function now starts to load and start the activity according to the informations<br /> it received from the am server.
The client side bindApplication function now starts to load and start the activity according to the informations<br />it received from the am server.


http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#683
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#683
Line 184: Line 150:
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/LoadedApk.java#479
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/LoadedApk.java#479


====6. App user code====
==== 6. App - user code ====


finally we get onCreate in our code
finally we get onCreate in our code
Line 190: Line 156:
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#869
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#869


====7. ActivityManager start====
==== 7. ActivityManager start ====


somewhere stuff happens through messages
somewhere stuff happens through messages


====5. Zygote show window====
==== 5. Zygote - show window ====


through magic messages this happens:
through magic messages this happens:
Line 203: Line 169:


http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#4959
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#4959
<code><br />mWindow = PolicyManager.makeNewWindow(this);<br /></code>


wew that was hard to find.
wew that was hard to find.


==Graphics stack==
== Graphics stack ==


Information gathered from a quick skimming of the sources
Information gathered from a quick skimming of the sources
Line 216: Line 184:
* which can be sent to eglCreateWindowSurface()
* which can be sent to eglCreateWindowSurface()
* holds a reference to SurfaceFlinger
* holds a reference to SurfaceFlinger
* has a <span class="caps">HWC</span>omposer&amp; getHWComposer() ?
* has a HWComposer&amp;amp; getHWComposer() ?
* lets you register a VSyncHandler callback object
* lets you register a VSyncHandler callback object
* single-process Qt should be easily doable using this approach
* single-process Qt should be easily doable using this approach
Line 223: Line 191:


* allocates graphics buffers
* allocates graphics buffers
* constructor does gralloc_open with a hw module from hw_get_module with <span class="caps">GRALLOC</span>_HARDWARE_MODULE_ID
* constructor does gralloc_open with a hw module from hw_get_module with GRALLOC_HARDWARE_MODULE_ID
* destructor does galloc_close
* destructor does galloc_close
* gralloc_open returns a device of type alloc_device_t
* gralloc_open returns a device of type alloc_device_t
Line 229: Line 197:
* and a free function, naturally
* and a free function, naturally


native/libs/ui/GraphicBuffer.cpp
native/libs/ui/GraphicBuffer.cpp<br />* has some code for serializing buffer_handle_t's (flatten / unflatten)<br />* locks / unlocks buffer_handle_t via GraphicBufferMapper which just forwards to the hw_get_module GRALLOC_HARDWARE_MODULE_ID, similar to GraphicsBufferAllocator
 
* has some code for serializing buffer_handle_t’s (flatten / unflatten)
* locks / unlocks buffer_handle_t via GraphicBufferMapper which just forwards to the hw_get_module <span class="caps">GRALLOC</span>_HARDWARE_MODULE_ID, similar to GraphicsBufferAllocator


native/libs/gui/BufferQueue.cpp
native/libs/gui/BufferQueue.cpp


* from the comments // BufferQueue manages a pool of gralloc memory slots to be used // by producers and consumers.
* from the comments<br /> // BufferQueue manages a pool of gralloc memory slots to be used<br /> // by producers and consumers.
* has a consumer and producer facing interface
* has a consumer and producer facing interface
* supports queuing / dequeuing / canceling buffers, draining the buffer queue, etc
* supports queuing / dequeuing / canceling buffers, draining the buffer queue, etc
* on the consumer side, acquiring and releasing buffers
* on the consumer side, acquiring and releasing buffers
* uses eglClientWaitSyncKHR() to synchronize
* uses eglClientWaitSyncKHR() to synchronize
* used by both <span class="caps">EGL</span>, camera, and media <span class="caps">API</span>s
* used by both EGL, camera, and media APIs


native/libs/gui/SurfaceTexture.cpp
native/libs/gui/SurfaceTexture.cpp


* has code for mapping GraphicBuffer’s to textures
* has code for mapping GraphicBuffer's to textures
* interacts with the buffer queue
* interacts with the buffer queue


Line 256: Line 221:
native/services/surfaceflinger/Layer.cpp
native/services/surfaceflinger/Layer.cpp


* used by SurfaceFlinger to actually render Surface’s (happens in Layer::onDraw)
* used by SurfaceFlinger to actually render Surface's (happens in Layer::onDraw)
* onDraw is called from base class LayerBase, LayerBase::draw or LayerBase::drawForSreenShot (lol)
* onDraw is called from base class LayerBase, LayerBase::draw or LayerBase::drawForSreenShot (lol)


Line 268: Line 233:
* starts sleep management on the DisplayHardware object
* starts sleep management on the DisplayHardware object
* triggers the boot animation
* triggers the boot animation
* does dirty region handling / partial updates (unlike <span class="caps">QML</span> scene graph)
* does dirty region handling / partial updates (unlike QML scene graph)
* composeSurfaces calls Layer::draw on each of the layers
* composeSurfaces calls Layer::draw on each of the layers


==Single-process non-composited Qt==
== Single-process non-composited Qt ==


Non-composited (i.e. not running surfaceflinger, but running Qt directly on the framebuffer with <span class="caps">EGL</span>/GLES2) is now possible with the following patch and its dependencies: https://codereview.qt.io/#change,38199
Non-composited (i.e. not running surfaceflinger, but running Qt directly on the framebuffer with EGL/GLES2) is now possible with the following patch and its dependencies: https://codereview.qt.io/#change,38199


To get a Qt application to run at start-up, you’ll want to edit system/core/rootdir/init.rc.
To get a Qt application to run at start-up, you'll want to edit system/core/rootdir/init.rc.


Comment out the surfaceflinger bits like so:
Comment out the surfaceflinger bits like so:
<code><br />#service surfaceflinger /system/bin/surfaceflinger<br /># class main<br /># user system<br /># group graphics<br /># onrestart restart zygote<br /></code>


And add a service right below for the Qt application you want to run, example:
And add a service right below for the Qt application you want to run, example:


After that, you need to run make in the top-level build directory, and re-flash boot.img
<code><br />service myqtapp /system/bin/qmlscene /data/someqml.qml -plugin evdevtouch:/dev/input/event0<br /> class main<br /> user system<br /> group graphics input<br /> setenv QT_QPA_EGLFS_NO_SURFACEFLINGER 1<br /> setenv QT_QPA_EGLFS_HIDECURSOR 1<br /> setenv QML_FORCE_THREADED_RENDERER 1<br /></code>

Revision as of 10:01, 24 February 2015

[toc align_right="yes&quot; depth="5&quot;]

Research

  1. Figuring out stuff for the lolz.
  2. ??
  3. ??
  4. Profit!

UI

Ok , we've got qml2 kicking on Surfaceflinger at 60FPS.
http://www.youtube.com/watch?v=TEkubKZaUt8

Input comes from evdev for now, until we figure out how to reuse androids ui management.

compositing multiple windows and stuff isn't clear yet either.

Services

Android is designed to expose services through the java public API. Since we're running native, reusing services is a mixed bag.
We need to find a process border that:

  • Keeps Qt and Android seperate things from a legal perspective.
  • Allows us to rip out services and replace them with other stuff.
  • Doesn't change too frequently upstream.

Let's see what we can do with just using binder directly.
II'll just show the exciting parts. You can read up all the interfaces form the aidl.

Since google likes to mess with method index ordering,
(for example commit 25101b0b9a84571ead15b26e9f4cd9c4298d7823 in frameworks/base.)
it's pretty clear that we need to parse the aidls to get the correct order instead of hardcoding them.
The numbers shown here are for 4.0.1

Telephony

frameworks/base/telephony/java/com/android/internal/telephony/ITelephony.aidl

Method Index prototype *effect*
0 void dial(String number); opens the dialer ui with the given number
1 void call(String number); calls the given number
3 endCall() guess what
4 answerRingingCall() Answer the phone when it rings
5 silenceRinger() Or don't
? boolean setRadio(boolean turnOn); switch on/off radio (flight mode)

We apparantly can get a notification for incomming calls from ITelephonyRegistry.aidl.
cool, but what if we want our own call ui?
Well, darn, those service interfaces are part of the ui.
So replacing the UI involves talking to RIL directly.

Connectivity Management

./frameworks/base/core/java/android/net/IConnectivityManager.aidl

Method Index prototype *effect*
2 NetworkInfo getActiveNetworkInfo(); i get nothing
12 boolean setRadios(boolean onOff); disables/enables all internet connections but leaves the actual radio emission on (not flight mode)
25 String[] getTetheredIfaces nothing

App Integration

Headless

<br />am start -a android.intent.action.MAIN -n com.android.browser/.BrowserActivity<br />am start -a android.intent.action.MAIN -n com.android.settings/.Settings<br />

wohoo magic. now let's try headless.
Looks like this was intended for Google Q, which has no UI at all.

<br />setprop ro.config.headless 1<br />stop zygote<br />start zygote<br />am start -a android.intent.action.MAIN <s>n com.android.browser/.BrowserActivity<br />


E/ActivityManager( 1733): Starting activities not supported on headless device: ActivityRecord{478c83e8 com.android.browser/.BrowserActivity}
oh well, worth a try. We'll start systemserver again and see how much we can remove without breaking apps.
| Service | started from | effect of removal |
| SystemUi | SystemServer.java | missing softbuttons and statusbar |
| WindowManager | SystemServer.java | Crashes. Systemserver has refs all over the place. |
| ActivityManager | SystemServer.java | Ah well, what did you expect… |
| Launcher | am/ActivityManagerService.java:2122 | Input stops working. can't get past lockscreen |
SystemUi is an app (frameworks/base/packages/SystemUI/src/com/android/systemui),
same for homescreen (./packages/apps/Launcher2/)
but that's about it. Everything else is systemserver, period.
h3. Application startup path
Android uses a Async technique to start applications. The Zygote process preloads all needed libraries and just
gets forked for every new started app, the client process then loads the apk and start activities or services.
h4. 1. Launcher
/frameworks/base/core/java/android/app/
First a user clicks/touches a shortcut on the android launcher,
launcher calls Activity::startActivity() which forwards the call to
ActivityManagerNative::startActivity()


<br /> int result = ActivityManagerNative.getDefault()<br /> .startActivity(whoThread, intent,<br /> intent.resolveTypeIfNeeded(who.getContentResolver()),<br /> token, target != null ? target.mEmbeddedID : null,<br /> requestCode, 0, null, null, options);<br />


ActivityManagerNative now executes a Binder transaction to the am Server.
h4. 2. ActivityManager
/frameworks/base/services/java/com/android/server/am
ActivityManagerNative now receives the binder call and executes the real startActivity implementation:
http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#2349
which calls :
# ActivityStack::startActivityMayWait
http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#2998
# ActivityStack::startActivityUncheckedLocked
http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#2384
# ActivityStack::startActivityLocked (overloaded)
http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1749
# ActivityStack::resumeTopActivityLocked
http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#1380
I assume this function is called for existing and not existing processes, the code is hard to read here
# ActivityStack::startSpecificActivityLocked
http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityStack.java#startSpecificActivityLocked
# ActivityManagerService::startProcessLocked
http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#1883
# ActivityManagerService::startProcessLocked (overload)
http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#1987


startProcessLocked will now execute the new Process:

<br />// Start the process. It will either succeed and return a result containing<br />// the PID of the new process, or else throw a RuntimeException.<br />Process.ProcessStartResult startResult = Process.start(&quot;android.app.ActivityThread&amp;quot;,<br /> app.processName, uid, uid, gids, debugFlags,<br /> app.info.targetSdkVersion, null);<br />


Process.start tells Zygote on its socket (/dev/socket/zygote) to fork and load ActivityThread
Zygote responds with the forked pid, which am puts into a list.


<br />this.mPidsSelfLocked.put(startResult.pid, app);<br />


From now on the currently created Process is responsible for getting back to the am server,
to register itself and ask for the needed informations about the application it has to start.
h4. 3. Zygote ActivityThread

/frameworks/base/core/java/android/app/

The newly created application enters the main function in ActivityThread:
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#4720

One of the first actions it has to do is talk back to the am-server that it has started and
is now ready to load the application.
This is done in the ActivityThread::attach() function:
http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#attach

<br />//get the default ActivityManager (am-server)<br />IActivityManager mgr = ActivityManagerNative.getDefault();<br />try {<br /> //tell the am server that we are ready to load the application<br /> mgr.attachApplication(mAppThread);<br />} catch (RemoteException ex) {<br /> //Ignore<br />}<br />

Control now goes back to the am-server to setup the process , mgr.attachApplication should block until the server is finished

4. ActivityManager attach

/frameworks/base/services/java/com/android/server/am

The function ActivityManagerService::attachApplicationLocked is executed

http://androidxref.com/4.1.1/xref/frameworks/base/services/java/com/android/server/am/ActivityManagerService.java#attachApplicationLocked

, it puts together all needed informations and calls the bindApplication func on the ActivityThread interface.

5. Zygote - load app

/frameworks/base/core/java/android/app/

The client side bindApplication function now starts to load and start the activity according to the informations
it received from the am server.

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#683

the app gets loaded from apkg

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/LoadedApk.java#479

6. App - user code

finally we get onCreate in our code

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#869

7. ActivityManager start

somewhere stuff happens through messages

5. Zygote - show window

through magic messages this happens:

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/ActivityThread.java#1950

which calls Activity.attach

http://androidxref.com/4.1.1/xref/frameworks/base/core/java/android/app/Activity.java#4959

<br />mWindow = PolicyManager.makeNewWindow(this);<br />

wew that was hard to find.

Graphics stack

Information gathered from a quick skimming of the sources

native/services/surfaceflinger/DisplayHardware/DisplayHardware.cpp

  • appears to allocate a FramebufferNativeWindow()
  • gets an ANativeWindow * from it
  • which can be sent to eglCreateWindowSurface()
  • holds a reference to SurfaceFlinger
  • has a HWComposer&amp; getHWComposer() ?
  • lets you register a VSyncHandler callback object
  • single-process Qt should be easily doable using this approach

native/libs/ui/GraphicBufferAllocator.cpp

  • allocates graphics buffers
  • constructor does gralloc_open with a hw module from hw_get_module with GRALLOC_HARDWARE_MODULE_ID
  • destructor does galloc_close
  • gralloc_open returns a device of type alloc_device_t
  • alloc_device_t has an alloc function that returns a buffer_handle_t given a width/height/usage/format/stride
  • and a free function, naturally

native/libs/ui/GraphicBuffer.cpp
* has some code for serializing buffer_handle_t's (flatten / unflatten)
* locks / unlocks buffer_handle_t via GraphicBufferMapper which just forwards to the hw_get_module GRALLOC_HARDWARE_MODULE_ID, similar to GraphicsBufferAllocator

native/libs/gui/BufferQueue.cpp

  • from the comments
    // BufferQueue manages a pool of gralloc memory slots to be used
    // by producers and consumers.
  • has a consumer and producer facing interface
  • supports queuing / dequeuing / canceling buffers, draining the buffer queue, etc
  • on the consumer side, acquiring and releasing buffers
  • uses eglClientWaitSyncKHR() to synchronize
  • used by both EGL, camera, and media APIs

native/libs/gui/SurfaceTexture.cpp

  • has code for mapping GraphicBuffer's to textures
  • interacts with the buffer queue

native/libs/gui/Surface.cpp

  • client side interface to SurfaceTexture
  • SurfaceControl controls how a surface is positioned, cropped, etc
  • Surface provides the EGLNativeWindowType

native/services/surfaceflinger/Layer.cpp

  • used by SurfaceFlinger to actually render Surface's (happens in Layer::onDraw)
  • onDraw is called from base class LayerBase, LayerBase::draw or LayerBase::drawForSreenShot (lol)

native/services/surfaceflinger/SurfaceFlinger.cpp

  • creates a DisplayHardware object
  • allocates a control block (for communication with clients?)
  • seems to only support one display at the time, has some comments about multiple displays
  • does makeCurrent and some GL initialization
  • starts an EventThread
  • starts sleep management on the DisplayHardware object
  • triggers the boot animation
  • does dirty region handling / partial updates (unlike QML scene graph)
  • composeSurfaces calls Layer::draw on each of the layers

Single-process non-composited Qt

Non-composited (i.e. not running surfaceflinger, but running Qt directly on the framebuffer with EGL/GLES2) is now possible with the following patch and its dependencies: https://codereview.qt.io/#change,38199

To get a Qt application to run at start-up, you'll want to edit system/core/rootdir/init.rc.

Comment out the surfaceflinger bits like so:

<br />#service surfaceflinger /system/bin/surfaceflinger<br /># class main<br /># user system<br /># group graphics<br /># onrestart restart zygote<br />

And add a service right below for the Qt application you want to run, example:

<br />service myqtapp /system/bin/qmlscene /data/someqml.qml -plugin evdevtouch:/dev/input/event0<br /> class main<br /> user system<br /> group graphics input<br /> setenv QT_QPA_EGLFS_NO_SURFACEFLINGER 1<br /> setenv QT_QPA_EGLFS_HIDECURSOR 1<br /> setenv QML_FORCE_THREADED_RENDERER 1<br />