Qt Script V8 Port

IMPORTANT: The content of this page is outdated. Reason: This project is abandoned If you have checked or updated this page and found the content to be suitable, please remove this notice.

This article may require cleanup to meet the Qt Wiki's quality standards. Reason: Auto-imported from ExpressionEngine. Please improve this article if you can. Remove the {{cleanup}} tag and add this page to Updated pages list after it's clean.

V8 is Google's open source JavaScript engine, used in the Google Chrome browser. The QtScript V8 port is an effort to implement the existing QtScript API on top of V8.

The master task for researching whether Qt (including QtWebKit) can be "switched over" to using V8 is QTBUG-12502.

Goal

The ideal outcome of this work is a new version of the QtScript library that QML, QtWebKit and Qt applications can readily benefit from (e.g., through improved script performance due to the V8 compiler technology).

We strive to preserve source, binary and behavioral with existing versions of QtScript, so that projects that already use QtScript can move to the V8-based version without much impact on their existing code.

Getting the Code

http://qt.gitorious.org/+qt-developers/qt/qt-script-ng

The repository is is a full clone of Qt that can be configured and built like usual. It includes a copy of V8-Isolates in src/3rdparty/v8, which is automatically built as part of building QtScript. master-v8 is the currently active branch.

Workflow

The port was started by removing the old QtScript implementation completely, and creating "stubs" (empty implementations) for all functions in the QtScript API. Thus, the starting point was a QtScript library that's fully source and binary compatible with previous QtScript releases. When an unimplemented API function is invoked, it will issue a "Not implemented" warning, and otherwise do nothing. This enables us to incrementally implement the API, while all the time being able to test the implementation against existing autotest suites and applications (including QML).

Our workflow consists of running an application/autotest and fixing failures, one at a time. Eventually, there will be no more failures, and no show-stopping performance regressions against previous QtScript releases, at which time the project can be considered "Done".

Test failures are tracked at QTBUG-17640.

About V8

Like QtScript, V8 provides a C++ API for embedding JavaScript into an application, and for integrating with native application code.

The V8 Embedder's Guide gives a good introduction to the V8 C++ API. If you plan to work on the QtScript port, it's a good idea to first familiarize yourself with the V8 design and concepts (handles, handle scopes, accessors, interceptors). The core API is all in one file in the V8 repository: include/v8.h. There are additional includes for e.g. debugging and profiling APIs.

The How to Download and Build V8 page has instructions for getting the V8 source code and building it. However, for starters you shouldn't have to worry about that, since the QtScript V8 port includes its own copy of V8 that's built as part of building QtScript.

If you want to follow V8 development closely, consider subscribing to the v8-dev list. This is a rather high-volume list, consisting mostly of automated mails from code reviews and commits. There's also a v8-users list for discussing how to use V8.

V8 Isolates

V8 Isolates is a research project by Google to support multiple V8 "instances" per process. This feature will enable Chromium to support in-process workers. Coincidentally, this feature is a requirement for the QtScript V8 port, since the QtScript API is re-entrant.

V8 Isolates is developed in the

experimental/isolates<code> branch in the V8 repository. Essentially, this branch gathers all static variables in V8 into thread-local storage. Isolates is the branch we track and import into QtScript.

Google aims to [http://code.google.com/p/v8/issues/detail?id=510#c15 eventually land Isolates on the main development branch].

== Relevant V8 Bugs ==

We use the [http://code.google.com/p/v8/issues/list V8 issue tracker] to report and track bugs relevant to QtScript, such as ECMA compliance issues, build issues, V8 API behavior, and suggestions for additional API. This section lists those issues.

* [http://code.google.com/p/v8/issues/detail?id=510 510]: Several V8 instances in a process
* [http://code.google.com/p/v8/issues/detail?id=1204 1204]: Limitation of setData method
* [http://code.google.com/p/v8/issues/detail?id=1205 1205]: Isolate::setData
* [http://code.google.com/p/v8/issues/detail?id=1021 1021]: Prototype Setter Interceptor not working
* [http://code.google.com/p/v8/issues/detail?id=1022 1022]: Interceptor "blocking" setter in prototype
* [http://code.google.com/p/v8/issues/detail?id=1028 1028]: RangeError in Array ctor (ECMA compliance)
* [http://code.google.com/p/v8/issues/detail?id=1035 -1035-]: Memory leak in Isolate class
* [http://code.google.com/p/v8/issues/detail?id=1036 -1036-]: Memory leak in Logger class
* [http://code.google.com/p/v8/issues/detail?id=884 884]: shell sample Infinite loop in JSObject::LookupCallbackSetterInPrototypes (''Possibly a gcc bug; watch out for it in release builds'')
* [http://code.google.com/p/v8/issues/detail?id=435 435]: The .length property of a function always returns zero (''actually, of native functions. Having this fixed would also provide one clean way to make the length parameter of QScriptEngine::newFunction() work'')
* [http://code.google.com/p/v8/issues/detail?id=1061 1061]: Some function not working in Isolate branch (work arounded in our patched V8)
* [http://code.google.com/p/v8/issues/detail?id=1072 1072]: TryCatch doesn't catch exceptions thrown from an accessor.
* [http://code.google.com/p/v8/issues/detail?id=1078 1078]: Assert when changing the prototype of the global object. (some test change the protoype of the global object)
* [http://code.google.com/p/v8/issues/detail?id=862 862]: GetOwnProperty() should return enumerable properties for String chars (''ECMA compliance item that we check in our autotests'')
* [http://code.google.com/p/v8/issues/detail?id=1256 -1256-]: v8::Array::New() ignores length argument (causes QScriptEngine::newArray(123) to be slow, since we have to set the length explicitly)
* [http://code.google.com/p/v8/issues/detail?id=831 831]: regexp performance seems to degrade exponentially with length of failed ([http://code.google.com/p/v8/issues/detail?id=1259 1259] has been merged into this issue)

== Reporting V8 Bugs ==

If you find a bug in V8 that affects QtScript, you should create a an upstream report in the [http://code.google.com/p/v8/issues/list V8 issue tracker]. It's important that the report has a minimal testcase, and in particular that it doesn't depend on QtScript; i.e., the testcase should use the V8 API directly.

Since we're working with a (not-necessarily-up-to-date) copy of V8 Isolates, it's possible that the bug isn't present in upstream V8 (because it's been fixed already). You should first try to reproduce the bug with V8 trunk or bleeding_edge, and report it straight away if it's present there. If you can't reproduce the bug in trunk or bleeding_edge, but can reproduce it in the Isolates branch, it's either because the fix hasn't been merged into Isolates yet, or there's an Isolates-specific bug. Depending on the nature of the bug, you'll have to try to decide which it is.

The purpose of the testcase is to make it easy for the V8 developers to reproduce the bug. Here's a template main.cpp:

1. include <v8.h>
2. include <assert.h>
3. include <stdio.h>

int main(int argc, char **argv) {

v8::V8::SetFlagsFromCommandLine(&argc, argv, true);

v8::HandleScope handle_scope;

v8::Handle<v8::ObjectTemplate> global = v8::ObjectTemplate::New();
// Put your global object properties here…
// global->Set(v8::String::New("foo"), v8::Integer::New(123));

v8::Persistent<v8::Context> context = v8::Context::New(NULL, global);

assert(!context.IsEmpty());
v8::Context::Scope context_scope(context);

v8::TryCatch try_catch;

// Put your script here…

v8::Handle<v8::String> source = v8::String::New("'hello'");
assert(!source.IsEmpty());

v8::Handle<v8::Script> script = v8::Script::Compile(source);

assert(!script.IsEmpty());

v8::Handle<v8::Value> result = script->Run();

assert(!result.IsEmpty());

// Print the result…

v8::String::Utf8Value str(result);
printf("%s\n", *str);

context.Dispose();

v8::V8::Dispose();
return 0;

}

Assuming you've built V8 in $V8DIR, here's how you can build it with GCC:

g++ -I$V8DIR/include -L$V8DIR -lv8 template.cpp -o mybug

Once the testcase is working, you can strip away everything that's not needed (asserts and such) before submitting.

Finally, add a link to the newly created upstream bug in the previous section ("Relevant V8 Bugs"), and preferably refer to the bug in our JIRA ("This depends on the following upstream bug: …"), and source code (e.g., update the QEXPECT_FAIL message of a test).

Hacking on the V8 Build

This section provides information about how we build V8; this is useful to know if you encounter V8-related compile-time or runtime issues, for example.

By default, QtScript uses a copy of V8 located in

src/3rdparty/v8<code>

V8 is built as a separate (static) library that QtScript links against. The default (upstream) build system for V8 is [http://www.scons.org/ SCons]. However, to avoid a dependency on SCons (which isn't available on all Qt platform — Symbian in particular), we currently maintain a QMake project file for building V8:

src/script/v8.pro This file is a (manual) port of (a subset of) the SConstruct and src/SConscript files in the V8 repository.

Currently, Linux and Mac OS are the only regularly tested platforms.

generated/libraries.cpp

V8 implements large parts of its functionality (such as the core ECMA objects) in JavaScript itself; see the .js files in the V8 src/ directory. V8 evaluates these files as part of "bootstrapping" the V8 environment. In order to avoid an external dependency on the .js files, their contents are embedded in the V8 library (analogous to how Qt resource (.qrc) files are used by Qt). A Python script in the V8 directory (tools/js2c.py) generates a C++ file which is literally a char array of the .js contents, plus functions for retrieving script contents by index/name.

The js2c.py script is automatically invoked when building V8, and will regenerate libraries.cpp if any of the V8 .js files are modified.

Snapshots

By default, we build V8 with snapshot support. This is a feature of V8 that greatly improves creation time of the V8 environment. When using snapshots, instead of evaluating the thousands of lines of JavaScript contained in the built-in .js files (see previous section), V8 will initialize itself from a serialized version. This is possible since, for a given build of V8 on a given platform/CPU, the state of V8 after evaluating the built-in .js files will always be the same.

mksnapshot

The snapshot feature is aided by a tool, mksnapshot, which creates a V8 environment the "normal" way (by evaluating .js files et al), and generates a C++ file (generated/snapshot.cpp) containing the serialized V8 state. This file is then linked into QtScript to provide V8 with the snapshot.

mksnapshot is automatically built and run when building QtScript.

Versioning/Compatibility

The serialized V8 state is platform/CPU-specific, and isn't guaranteed to work across V8 versions (or the same V8 version compiled with different defines/compiler options). V8 has some code to check whether the snapshot data is actually useable; if it isn't, you'll hopefully get a failing assert during deserialization (rather than some obscure crash later).

Cross-compiling

When cross-compiling, snapshots are disabled, since the mksnapshot tool would have to be run on the target system in order to generate the snapshot. (You could try generating it in an emulator, but the emulator would have to be perfectly compatible with the target — note that V8 detects certain CPU features at run time, and that the snapshot can contain native code that relies on certain features (instructions) being supported. If they aren't, undefined behavior (typically, a crash) will ensue.)

When deploying QtScript to devices, the snapshot would have to be generated in such a way that it's compatible with any supported device/configuration (the "lowest common denominator" — e.g. ARMv5, VFPv2, …).

Manually Disabling Snapshots

The snapshot feature is not enabled by default upstream, so it's perhaps not as well-tested as non-snapshot builds. If you encounter strange crashes in V8 code (typically at construction time), you can disable snapshots to quickly check whether that's the issue.

To disable snapshots, set the QMake variable V8SNAPSHOT to "no". E.g.

qmake -r "V8SNAPSHOT=no"<code>

and rebuild QtScript.

==== Dynamic Snapshotting? ====

It would be awesome if, rather than being a compile-time step, V8 would create and use snapshots dynamically; e.g. the first time a V8 environment is created, a snapshot is automatically created, and subsequent V8 environments would be initialized from the snapshot.

=== Updating src/3rdparty/v8 ===

Updating Qt's copy of V8 is done using the util/scripts/mkdist-v8 script. It takes a git-svn checkout of V8 and copies a specified revision of it to src/3rdparty/v8.

Before running the script, you first need to set the git v8.url variable to point to your V8 checkout:

git config v8.url path/to/v8

Then, in the V8 repository, create a tag that references the SHA1 you want to import (e.g. HEAD):

git tag v8-snapshot-23032011<code>

Lastly, in the Qt source tree, run

util/scripts/mkdist-v8 <tag>

Everything will be staged, and you can run

git commit

to commit it all. But before you do that, you probably want to try to recompile QtScript.

Fixing the build

If .cc files have been added, you must manually add them to src/script/v8/v8.pro.

You should try the build in both debug and release mode. If you get errors, try compiling V8 the normal way outside of Qt. Pay attention to the defines that are passed to the compiler; if there's a define missing for the QtScript/V8 build, add it to src/script/v8/v8.pri. (Note that the upstream V8 might not have been tested with(out) the defines we use; make sure that it's not a bug in V8, e.g. that the upstream V8 builds without ENABLE_LOGGING_AND_PROFILING defined (pass "profilingsupport=off" to SCons).)

Excluding files