D-Pointer
English 简体中文 Українська Български Español فارسی
What is the d-pointer
If you have looked into Qt source files like this one [qt.gitorious.org], you will find it generously sprinkled with
Q_D
and
Q_Q
macros. This article unravels the purpose of these macros. The
Q_D
and
Q_Q
macros are part of a design pattern called the d-pointer (also called the opaque pointer [en.wikipedia.org]) where the implementation details of a library may be hidden from its users and changes to the implementation can be made to a library without breaking binary compatibility.
Binary compatibility, what is that?
When designing libraries like Qt, it is desirable that applications that dynamically link to Qt continue to run without recompiling even after the Qt library is upgraded/replaced with another version. For example, if your application CuteApp was based on Qt 4.5, you should be able to upgrade the Qt libraries (on Windows shipped with the application, on Linux often comes from package manager automatically!) from version 4.5 to Qt 4.6 and your CuteApp that was built with Qt 4.5 should still be able to run.
What breaks binary compatibility
So, when does a change in the library require a recompilation of the application? Let’s take a simple example:
Here we have a Widget that contains geometry as a member variable. We compile our Widget and ship it as WidgetLib 1.0.
For WidgetLib 1.1, someone comes up with the bright idea to add support for stylesheets. No sweat, we just add new methods and add a new data member.
We ship WidgetLib 1.1 with the above change only to find that CuteApp that was compiled and ran just fine with WidgetLib 1.0 crashes gloriously!
Why did it crash?
The reason is that by adding a new data member, we ended up changing the size of Widget and Label objects. Why does this matter? When your C++ compiler generates code, it uses ‘offsets’ to access data within an object.
Here’s an oversimplified version of how the above POD objects might look in memory.
| Label object layout in WidgetLib 1.0 | Label object layout in WidgetLib 1.1 |
| m_geometry <offset 0> | m_geometry <offset 0> |
| ——————— | m_stylesheet <offset 1> |
| m_text <offset 1> | ————————- |
| ———————- | m_text <offset 2> |
In WidgetLib 1.0, the text member of Label was at (logical) offset 1. The code generated by the compiler in the application for the method
Label::text()
translates to accessing offset 1 of the label object in the application. In WidgetLib 1.1, the ‘‘text’‘ member of Label has shifted to (logical) offset 2! Since the application has not been recompiled, it continues to think that
text
is at offset 1 and ends up accessing the
stylesheet
variable! I am sure at this point there are a few who are wondering why the
Label::text()
‘s offset calculation code ended up in the CuteApp binary and not in the WidgetLib binary. The answer is that the code for
Label::text()
was defined in the header file and the compiler ended up inlining [en.wikipedia.org] it. So, does the situation change if
Label::text()
had not been inlined? Say,
Label::text()
was moved to the source file? Well, no. The C++ compiler relies on the size of objects being the same at compile time and run-time. For example, stack winding/unwinding – if you created a Label object on the stack, the compiler generated code to allocate space on the stack based on the Label’s size at compile time. Since the size of Label is different at run time in WidgetLib 1.1, Label’s constructor overwrites existing stack data and ends up corrupting the stack.
Never change the size of an exported C++ class
In summary, never ever change the size or layout (don’t move the data around) of exported (i.e visible to the user) C++ classes once your library has been released. A C++ compiler generates code assuming that the size or the ordering of data in a class does not change after the application has been compiled.
So, how can one not change the size of the object and yet add new features?
The d-pointer
The trick is to keep the size of all public classes of a library constant by only storing a single pointer. This pointer points to a private/internal data structure that contains all the data. The size of this internal structure can shrink or grow without having any side-effect on the application because the pointer is accessed only in the library code and from the application’s point of view the size of the object never changes – it’s always the size of the pointer. This pointer is called the d-pointer.
The spirit of this pattern is outlined in the code below (all code in this article doesn’t have destructors, of course you should add them in real code).
widget.h
widget_p.h, which is the private header file of the widget class
widget.cpp
Next, there’s an example of a child class based on Widget.
label.h
label.cpp
With the above structure, CuteApp never accesses the d-pointer directly. And since the d-pointer is only ever accessed in WidgetLib and WidgetLib is recompiled for every release, the Private class can freely change with no impact on CuteApp.
Other benefits of d-pointer
It’s not all about binary compatibility. The d-pointer has other benefits:
- Hide implementation details – We can ship WidgetLib with just the header files and the binaries. The .cpp files can be closed source.
- The header file is clean of implementation details and can serve as the API reference.
- Since the header files needed for implementation are moved from the header file into the implementation (source) file, compiles are much faster.
It is indeed true that the above benefits appear trivial. The real reason to use d-pointers in Qt is for binary compatibility and the fact that Qt started out closed source.
The q-pointer
So far, we have only seen the d-pointer as a C-style data structure. In reality, it contains private methods (helper functions). For example,
LabelPrivate
might have a
getLinkTargetFromPoint()
helper function that is required to find the link target when the mouse is clicked. In many cases, these helper methods require access to the public class i.e some functions from Label or from its base class Widget. For example, a helper method,
setTextAndUpdateWidget()
might want to call
Widget::update()
which is a public method to schedule a redraw the Widget. So, the
WidgetPrivate
stores a pointer to the public class called the q-pointer. Modifying the code above for the q-pointer, we get:
widget.h
widget_p.h
widget.cpp
Next, another class based on Widget.
label.h
label.cpp
Inheriting d-pointers for optimization
In the above code, creating a single Label results in the memory allocation for
LabelPrivate
and
WidgetPrivate
. If we were to employ this strategy for Qt, the situation becomes quite worse for classes like
QListWidget
– it is 6 levels deep in the class inheritance hierarchy and it would result in upto 6 memory allocations!
This is solved by having an inheritance hierarchy for our private classes and having the class getting instantiated pass on a the d-pointer all the way up.
Notice that when inheriting d-pointers, the declaration of the private class has to be in a separate file, for example widget_p.h. It’s no longer possible to declare it in the widget.cpp file.
widget.h
widget_p.h
widget.cpp
label.h
label.cpp
Do you see the beauty? When we now create a
Label
object, it will create a
LabelPrivate
(which subclasses
WidgetPrivate
). It passes on the concrete d-pointer to Widget’s protected constructor! Now, when a
Label
object is created, there is only one memory allocation. Label also has a protected constructor that can be used by its subclasses to provide their own private classes.
d-pointers in Qt
In Qt, practically every public class uses the d-pointer approach. The only cases where it’s not used is when it is known in advance that the class will never ever have extra member variables added to it. For example, for classes like
QPoint
,
QRect
, no new members are expected to be added and hence the data members are stored straight into the class itself instead of using the d-pointer. Notice that in Qt, the base class of all Private objects is
QObjectPrivate
.
Q_D and Q_Q
A side effect of the optimization that we did in the previous step is that the q-ptr and d-ptr are of type
Widget
and
WidgetPrivate
. This means that the following won’t work.
Hence, when accessing the d-pointer in a subclass, we need to static_cast to the appropriate type.
As you can see, it’s not pretty having static_cast all over the place. Instead, there are two macros defined in src/corelib/global/qglobal.h which make it straighforward:
global.h
label.cpp
Q_DECLARE_PRIVATE and Q_DECLARE_PUBLIC
Qt classes have a
Q_DECLARE_PRIVATE
a macro in the public class. The macro reads:
qglobal.h
This macro can be used this way:
qlabel.h
The idea is that
QLabel
provides a function
d_func()
that allows access to its private internal class. The method itself is private (since the macro is inside a private section in qlabel.h). The
d_func()
can however be invoked by friends (C++ friend) of
QLabel
. This is primarily useful for access of information by Qt classes which cannot get access of some
QLabel
information using public api. As a bizarre example,
QLabel
might keep track of how many times the user has clicked on a link. However, there is no public API to access this information.
QStatistics
is a class that needs this information. A Qt developer will add
QStatistics
as a friend of
QLabel
and
QStatistics
can then do
label->d_func()->linkClickCount
. The
d_func
also has the advantage to enforce const-correctness: In a const member function of MyClass you need a Q_D(const MyClass) and thus you can only call const member functions in MyClassPrivate. With a “naked” d_ptr you could also call non-const functions. There’s also a
Q_DECLARE_PUBLIC
which goes the other way round.