<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.3.2">Jekyll</generator><link href="https://habkost.net/feed.xml" rel="self" type="application/atom+xml" /><link href="https://habkost.net/" rel="alternate" type="text/html" /><updated>2026-03-01T05:54:37+00:00</updated><id>https://habkost.net/feed.xml</id><title type="html">Eduardo Habkost</title><subtitle>Software Developer. C; QEMU; KVM; virtualization; Linux; Python; etc.
</subtitle><author><name>Eduardo Habkost</name></author><entry><title type="html">The long story of the query-cpu-model-expansion QEMU interface</title><link href="https://habkost.net/posts/2017/03/qemu-cpu-model-probing-story.html" rel="alternate" type="text/html" title="The long story of the query-cpu-model-expansion QEMU interface" /><published>2017-03-06T17:00:00+00:00</published><updated>2017-03-06T17:00:00+00:00</updated><id>https://habkost.net/posts/2017/03/qemu-cpu-model-probing-story</id><content type="html" xml:base="https://habkost.net/posts/2017/03/qemu-cpu-model-probing-story.html"><![CDATA[<p>So, finally the <a href="https://github.com/qemu/qemu/commit/666095c852d32df65b5982fcc8c85332979b7fc1"><code>query-cpu-model-expansion</code> x86 implementation was merged
into qemu.git</a>,
just before 2.9 soft freeze. Jiri Denemark already implemented
the x86 libvirt code to use it. I just can’t believe this was
finally done after so many years.</p>

<!--more-->

<p>It was a weird journey. It started almost 6 years ago with this
message to qemu-devel:</p>

<blockquote>
  <p>Date: Fri, 10 Jun 2011 18:36:37 -0300<br />
Subject: <a href="https://www.mail-archive.com/kvm@vger.kernel.org/msg55640.html">semantics of “-cpu host” and “check”/”enforce”</a></p>
</blockquote>

<p>…it continued on an interesting thread:</p>

<blockquote>
  <p>Date: Tue, 6 Mar 2012 15:27:53 -0300<br />
Subject: <a href="http://www.mail-archive.com/qemu-devel@nongnu.org/msg100533.html">Qemu, libvirt, and CPU models</a></p>
</blockquote>

<p>…on another very long one:</p>

<blockquote>
  <p>Date: Fri, 9 Mar 2012 17:56:52 -0300<br />
Subject: <a href="http://www.mail-archive.com/qemu-devel@nongnu.org/msg101215.html">Re: [Qemu-devel] [libvirt] Modern CPU models cannot be used with libvirt</a></p>
</blockquote>

<p>…and this one:</p>

<blockquote>
  <p>Date: Thu, 21 Feb 2013 11:58:18 -0300<br />
Subject: <a href="http://www.mail-archive.com/qemu-devel@nongnu.org/msg156534.html">libvirt&lt;-&gt;QEMU interfaces for CPU models</a></p>
</blockquote>

<p>I don’t even remember how many different interfaces were proposed
to provide what libvirt needed.</p>

<p>We had a few moments where we hopped back and forth between “just
let libvirt manage everything” to “let’s keep this managed by
QEMU”.</p>

<p>We took a while to get the QEMU community to decide how machine-type
compatibility was supposed to be handled, and what to do
with the weird CPU model config file we had.</p>

<p>The conversion of CPUs to QOM was fun. I think it started in 2012
and was finished only in 2015. We thought QOM properties would
solve all our problems, but then we found out that machine-types
and global properties make the problem more complex. The existing
interfaces would require making libvirt re-run QEMU multiple
times to gather all the information it needed. While doing the
QOM work, we spent some time fixing or working around issues with
global properties, qdev “static” properties and QOM “dynamic”
properties.</p>

<p>In 2014, my focus was moved to machine-types, in the hope that we
could finally expose machine-type-specific information to libvirt
without re-running QEMU. Useful code refactoring was done for
that, but in the end we never added the really relevant
information to the <code>query-machines</code> QMP command.</p>

<p>In the meantime, we had the
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1199446">fun TSX issues</a>,
and QEMU developers finally agreed to keep a few constraints on CPU
model changes, that would make the problem a bit simpler.</p>

<p>In 2015 IBM people started sending patches related to CPU models
in s390x. We finally had a multi-architecture effort to make CPU
model probing work. The work started by extending
<code>query-cpu-definitions</code>, but it was not enough. In June 2016 they
proposed a <code>query-cpu-model-expansion</code> API. It was finally merged
in September 2016.</p>

<p>I sent v1 of <code>query-cpu-model-expansion</code> for x86 in December 2016.
After a few rounds of reviews, there was a proposal to use
“-cpu max” to represent the “all features supported by this QEMU
binary on this host”. v3 of the series was
<a href="https://github.com/qemu/qemu/commit/666095c852d32df65b5982fcc8c85332979b7fc1">merged last week</a>.</p>

<p>I still can’t believe it finally happened.</p>

<p>Special thanks to:</p>
<ul>
  <li>Igor Mammedov, for all the x86 QOM/properties work and all the
valuable feedback.</li>
  <li>David Hildenbrand and Michael Mueller, for moving forward the
API design and the s390x implementation.</li>
  <li>Jiri Denemark, for the libvirt work, valuable discussions and
design feedback, and for the patience during the process.</li>
  <li>Daniel P. Berrangé, for the valuable feedback and for helping
making QEMU developers listen to libvirt developers.</li>
  <li>Andreas Färber, for the work as maintainer of QOM and the CPU
core, for leading the QOM conversion effort, and all the
valuable feedback.</li>
  <li>Markus Armbruster and Paolo Bonzini, for valuable feedback on
design discussions.</li>
  <li>Many others that were involved in the effort.</li>
</ul>]]></content><author><name>Eduardo Habkost</name></author><category term="virt" /><summary type="html"><![CDATA[So, finally the query-cpu-model-expansion x86 implementation was merged into qemu.git, just before 2.9 soft freeze. Jiri Denemark already implemented the x86 libvirt code to use it. I just can’t believe this was finally done after so many years.]]></summary></entry><entry><title type="html">Video and slides for FOSDEM 2017 talk: QEMU Internal APIs</title><link href="https://habkost.net/posts/2017/03/fosdem-2017-video-slides.html" rel="alternate" type="text/html" title="Video and slides for FOSDEM 2017 talk: QEMU Internal APIs" /><published>2017-03-06T00:00:00+00:00</published><updated>2017-03-06T00:00:00+00:00</updated><id>https://habkost.net/posts/2017/03/fosdem-2017-video-slides</id><content type="html" xml:base="https://habkost.net/posts/2017/03/fosdem-2017-video-slides.html"><![CDATA[<p>The slides and videos for my FOSDEM 2017 talk (<i>QEMU: internal APIs and conflicting world views</i>)
are available online.</p>

<ul>
  <li>Slides: <a href="https://habkost.net/talks/fosdem-2017/">https://habkost.net/talks/fosdem-2017/</a></li>
  <li>Video: <a href="https://fosdem.org/2017/schedule/event/iaas_qemintapi/">https://fosdem.org/2017/schedule/event/iaas_qemintapi/</a></li>
</ul>

<p>The subject I tried to cover is large for a 40-minute talk, but I
think I managed to scratch its surface and give useful examples.</p>

<p>Many thanks for the FOSDEM team of volunteers for the wonderful
event.</p>]]></content><author><name>Eduardo Habkost</name></author><category term="virt" /><summary type="html"><![CDATA[The slides and videos for my FOSDEM 2017 talk (QEMU: internal APIs and conflicting world views) are available online. Slides: https://habkost.net/talks/fosdem-2017/ Video: https://fosdem.org/2017/schedule/event/iaas_qemintapi/ The subject I tried to cover is large for a 40-minute talk, but I think I managed to scratch its surface and give useful examples. Many thanks for the FOSDEM team of volunteers for the wonderful event.]]></summary></entry><entry><title type="html">QEMU APIs: introduction to QemuOpts</title><link href="https://habkost.net/posts/2016/12/qemu-apis-qemuopts.html" rel="alternate" type="text/html" title="QEMU APIs: introduction to QemuOpts" /><published>2016-12-22T00:00:00+00:00</published><updated>2016-12-22T00:00:00+00:00</updated><id>https://habkost.net/posts/2016/12/qemu-apis-qemuopts</id><content type="html" xml:base="https://habkost.net/posts/2016/12/qemu-apis-qemuopts.html"><![CDATA[<p>This post is a short introduction to the QemuOpts API inside
QEMU. This is part of a series, see <a href="/posts/2016/11/incomplete-list-of-qemu-apis.html">the introduction</a> for
other pointers and additional information.</p>

<!--more-->

<p>QemuOpts was <a href="https://github.com/qemu/qemu/commit/e27c88fe9eb26648e4fb282cb3761c41f06ff18a">introduced in 2009</a>. It is a simple
abstraction that handles two tasks:</p>

<ol>
  <li><em>Parsing</em> of config files and command-line options</li>
  <li><em>Storage</em> of configuration options</li>
</ol>

<h2 id="data-structures">Data structures</h2>

<p>The QemuOpts data model is pretty simple:</p>

<ul>
  <li>
    <p><code>QemuOptsList</code> carries the list of all options belonging to a
given <em>config group</em>. Each entity is represented by a
<code>QemuOpts</code> struct.</p>
  </li>
  <li>
    <p><code>QemuOpts</code> represents a set of key-value pairs. (Some of the
code refers to that as a <em>config group</em>, but to avoid confusion
with <code>QemuOptsList</code>, I will call them <em>config sections</em>).</p>
  </li>
  <li>
    <p><code>QemuOpt</code> is a single key=value pair.</p>
  </li>
</ul>

<p>Some config groups have multiple <code>QemuOpts</code> structs (e.g.
“drive”, “object”, “device”, that represent multiple drives,
multiple objects, and multiple devices, respectively), while
others always have only one <code>QemuOpts</code> struct (e.g. the “machine”
config group).</p>

<p>For example, the following command-line options:</p>

<pre><code>-drive id=disk1,file=disk.raw,format=raw,if=ide \
-drive id=disk2,file=disk.qcow2,format=qcow2,if=virtio \
-machine usb=on -machine accel=kvm
</code></pre>

<p>are represented internally as:</p>

<p><img alt="Diagram showing two QemuOptsList objects: qemu_drive_opts and qemu_machine_opts. qemu_drive_opts has two QemuOpts entries: disk1 and disk2. disk2 has three QemuOpt entries: file=disk.raw, format=raw, if=ide. disk2 has three QemuOpt entries: file=disk.qcow2, format=qcow2, if=virtio. qemu_machine_opts has one QemuOpts entry. The QemuOpts entry for qemu_machine_opts has two QemuOpt entries: usb=on, accel=kvm" src="/qemuopts-example.mmd.png" /></p>

<h2 id="data-types">Data Types</h2>

<p>QemuOpts supports a limited number of data types for option
values:</p>

<ul>
  <li>Strings</li>
  <li>Boolean options</li>
  <li>Numbers (integers)</li>
  <li>Sizes</li>
</ul>

<h3 id="strings">Strings</h3>

<p>Strings are just used as-is, after the command-line or config
file is parsed.</p>

<p><em>Note:</em> On the command-line, options are separated by commas, but
commas inside option values can be escaped as <code>,,</code>.</p>

<h3 id="boolean-options">Boolean options</h3>

<p>The QemuOpt parser accepts only “on” and “off” as values for this
option.</p>

<p><strong>Warning:</strong> note that this behavior is different from the QOM
property parser. I plan to explore this in future posts.</p>

<h3 id="numbers-integers">Numbers (integers)</h3>

<p>Numbers are supposed to be unsigned 64-bit integers. However, the
code relies on the behavior of <code>strtoull()</code> and does <strong>not</strong>
reject negative numbers. That means the parsed <code>uint64_t</code> value
might be converted to a signed integer later. For example, the
following command-line is not rejected by QEMU:</p>

<pre><code>$ qemu-system-x86_64 -smp cpus=-18446744073709551615,cores=1,threads=1
</code></pre>

<p>I don’t know if there is existing code that requires negative
numbers to be accepted by the QemuOpts parser. I assume it
exists, so we couldn’t easily change the existing parsing rules
without breaking existing code.</p>

<h3 id="sizes">Sizes</h3>

<p>Sizes are represented internally as integers, but the parser
accept <em>suffixes</em> like <em>K</em>, <em>M</em>, <em>G</em>, <em>T</em>.</p>

<pre><code>qemu-system-x86_64 -m size=2G
</code></pre>

<p>is equivalent to:</p>

<pre><code>qemu-system-x86_64 -m size=2048M
</code></pre>

<p><em>Note:</em> there are two different size-suffix parsers inside QEMU:
<a href="https://github.com/qemu/qemu/blob/v2.8.0/util/cutils.c#L208">one at util/cutils.c</a> and
<a href="https://github.com/qemu/qemu/blob/v2.8.0/util/qemu-option.c#L177">another at util/qemu-option.c</a>.
Figuring out which one is going to be used is left as an exercise to the reader.</p>

<h2 id="working-around-the-qemuopts-parsers">Working around the QemuOpts parsers</h2>

<p>QEMU code sometimes uses tricks to avoid or work around the
QemuOpts option value parsers:</p>

<h3 id="example-1-using-the-raw-option-value">Example 1: using the raw option value</h3>

<p>It is possible to get the original raw option value as a string
using <code>qemu_opt_get()</code>, even after it was already parsed. For
example, the code that handles memory options in QEMU does that,
to ensure a suffix-less number is interpreted as Mebibytes, not
bytes:</p>

<figure class="highlight"><pre><code class="language-c" data-lang="c">    <span class="n">mem_str</span> <span class="o">=</span> <span class="n">qemu_opt_get</span><span class="p">(</span><span class="n">opts</span><span class="p">,</span> <span class="s">"size"</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="n">mem_str</span><span class="p">)</span> <span class="p">{</span>
        <span class="cm">/* [...] */</span>
        <span class="n">sz</span> <span class="o">=</span> <span class="n">qemu_opt_get_size</span><span class="p">(</span><span class="n">opts</span><span class="p">,</span> <span class="s">"size"</span><span class="p">,</span> <span class="n">ram_size</span><span class="p">);</span>
        <span class="cm">/* Fix up legacy suffix-less format */</span>
        <span class="k">if</span> <span class="p">(</span><span class="n">g_ascii_isdigit</span><span class="p">(</span><span class="n">mem_str</span><span class="p">[</span><span class="n">strlen</span><span class="p">(</span><span class="n">mem_str</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]))</span> <span class="p">{</span>
            <span class="n">sz</span> <span class="o">&lt;&lt;=</span> <span class="mi">20</span><span class="p">;</span>
            <span class="cm">/* [...] */</span>
        <span class="p">}</span>
    <span class="p">}</span></code></pre></figure>

<h3 id="example-2-empty-option-name-list">Example 2: empty option name list</h3>

<p>Some options do not use the QemuOpts value parsers at all, by not
defining any option names in the QemuOptsList struct. In those
cases, the option values are parsed and validated using different
methods. Some examples:</p>

<figure class="highlight"><pre><code class="language-c" data-lang="c"><span class="k">static</span> <span class="n">QemuOptsList</span> <span class="n">qemu_machine_opts</span> <span class="o">=</span> <span class="p">{</span>
    <span class="p">.</span><span class="n">name</span> <span class="o">=</span> <span class="s">"machine"</span><span class="p">,</span>
    <span class="p">.</span><span class="n">implied_opt_name</span> <span class="o">=</span> <span class="s">"type"</span><span class="p">,</span>
    <span class="p">.</span><span class="n">merge_lists</span> <span class="o">=</span> <span class="nb">true</span><span class="p">,</span>
    <span class="p">.</span><span class="n">head</span> <span class="o">=</span> <span class="n">QTAILQ_HEAD_INITIALIZER</span><span class="p">(</span><span class="n">qemu_machine_opts</span><span class="p">.</span><span class="n">head</span><span class="p">),</span>
    <span class="p">.</span><span class="n">desc</span> <span class="o">=</span> <span class="p">{</span>
        <span class="cm">/*
         * no elements =&gt; accept any
         * sanity checking will happen later
         * when setting machine properties
         */</span>
        <span class="p">{</span> <span class="p">}</span>
    <span class="p">},</span>
<span class="p">};</span></code></pre></figure>

<figure class="highlight"><pre><code class="language-c" data-lang="c"><span class="k">static</span> <span class="n">QemuOptsList</span> <span class="n">qemu_acpi_opts</span> <span class="o">=</span> <span class="p">{</span>
    <span class="p">.</span><span class="n">name</span> <span class="o">=</span> <span class="s">"acpi"</span><span class="p">,</span>
    <span class="p">.</span><span class="n">implied_opt_name</span> <span class="o">=</span> <span class="s">"data"</span><span class="p">,</span>
    <span class="p">.</span><span class="n">head</span> <span class="o">=</span> <span class="n">QTAILQ_HEAD_INITIALIZER</span><span class="p">(</span><span class="n">qemu_acpi_opts</span><span class="p">.</span><span class="n">head</span><span class="p">),</span>
    <span class="p">.</span><span class="n">desc</span> <span class="o">=</span> <span class="p">{</span> <span class="p">{</span> <span class="mi">0</span> <span class="p">}</span> <span class="p">}</span> <span class="cm">/* validated with OptsVisitor */</span>
<span class="p">};</span></code></pre></figure>

<p>This is a common pattern when options are translated to other
data representations: mostly <em>QOM properties</em> or <em>QAPI structs</em>.
I plan to explore this in a future blog post.</p>

<p>The following config groups use this method and do their own
parsing/validation of config options: <em>acpi</em>, <em>device</em>, <em>drive</em>,
<em>machine</em>, <em>net</em>, <em>netdev</em>, <em>numa</em>, <em>object</em>, <em>smbios</em>, <em>tpmdev</em></p>

<h2 id="-writeconfig">-writeconfig</h2>

<p>The QemuOpts code is responsible for two tasks:</p>

<ol>
  <li><strong>Parsing</strong> command-line options and config files</li>
  <li><strong>Storage</strong> of configuration options</li>
</ol>

<p>This means sometimes config options are parsed by custom code and
converted to QemuOpts data structures. Storage of config options
inside QemuOpts allow the existing QEMU configuration to be
written to a file using the <code>-writeconfig</code> command-line option.</p>

<p>The <a href="https://github.com/qemu/qemu/commit/715a664ac4ca3b9e44ffbc0ca41ecd91fbe96656">original commit</a> introducing
<code>-writeconfig</code> describes it this way:</p>

<blockquote>
  <p>In theory you should be able to do:</p>

  <pre><code>qemu &lt; machine config cmd line switches here &gt; -writeconfig vm.cfg
qemu -readconfig vm.cfg
</code></pre>

  <p>In practice it will not work.  Not all command line switches are
converted to QemuOpts, so you’ll have to keep the not-yet converted ones
on the second line.  Also there might be bugs lurking which prevent even
the converted ones from working correctly.</p>
</blockquote>

<p>This has improved over the years, but the comment still applies
today: most command-line options are converted to QemuOpts
options, but not all of them.</p>

<h2 id="further-reading">Further reading</h2>

<ul>
  <li><a href="https://github.com/qemu/qemu/blob/v2.8.0/vl.c#L3047">Main QemuOptsList registration code</a></li>
  <li>QemuOpts code: <a href="https://github.com/qemu/qemu/blob/v2.8.0/util/qemu-option.c">qemu-option.c</a>,
<a href="https://github.com/qemu/qemu/blob/v2.8.0/util/qemu-config.c">qemu-config.c</a></li>
  <li><a href="https://github.com/qemu/qemu/blob/v2.8.0/qapi/opts-visitor.c">opts-visitor.c</a>:
used when converting QemuOpts to QOM properties or QAPI structures</li>
</ul>]]></content><author><name>Eduardo Habkost</name></author><category term="virt" /><summary type="html"><![CDATA[This post is a short introduction to the QemuOpts API inside QEMU. This is part of a series, see the introduction for other pointers and additional information.]]></summary></entry><entry><title type="html">An incomplete list of QEMU APIs</title><link href="https://habkost.net/posts/2016/11/incomplete-list-of-qemu-apis.html" rel="alternate" type="text/html" title="An incomplete list of QEMU APIs" /><published>2016-11-29T02:51:00+00:00</published><updated>2016-11-29T02:51:00+00:00</updated><id>https://habkost.net/posts/2016/11/incomplete-list-of-qemu-apis</id><content type="html" xml:base="https://habkost.net/posts/2016/11/incomplete-list-of-qemu-apis.html"><![CDATA[<p>Having seen many people (including myself) feeling confused about
the purpose of some QEMU’s internal APIs when
reviewing and contributing code to QEMU, I am trying to document
things I learned about them.</p>

<!--more-->

<p>I want to make more detailed blog posts about some of them,
stating their goals (as I perceive them), where they are used,
and what we can expect to see happening to them in the future.
When I do that, I will update this post to include pointers to
the more detailed content.</p>

<h2 id="qemuopts">QemuOpts</h2>

<p><a href="https://github.com/qemu/qemu/commit/e27c88fe9eb26648e4fb282cb3761c41f06ff18a">Introduced in 2009</a>.
Compared to the newer abstractions below, it is quite simple. As
described in the original commit: it <em>“stores device parameters
in a better way than unparsed strings”</em>. It is still used by
configuration and command-line parsing code.</p>

<p>Making QemuOpts work with the more modern abstractions (esp. QOM
and QAPI) may be painful. Sometimes you can pretend it is not
there, but you can’t run away from it if you are dealing with
QEMU configuration or command-line parameters.</p>

<p><strong>See also:</strong> the <a href="/posts/2016/12/qemu-apis-qemuopts.html">Introduction to QemuOpts</a> blog post.</p>

<h2 id="qdev">qdev</h2>

<p>qdev <a href="https://github.com/qemu/qemu/commit/aae9460e244c7abe70b72ff374b3aa102bb09691">was added to QEMU in 2009</a>.
qdev manages the QEMU <em>device tree</em>, based on a hierarchy of
<em>buses</em> and <em>devices</em>. You can see the device tree managed by
qdev using the <code>info qtree</code> monitor command in QEMU.</p>

<p>qdev allows device code to register implementations of <em>device
types</em>. Machine code, on the other hand, would instantiate those
devices and configure them by setting <em>properties</em>, and not
accessing internal device data structures directly. Some devices
can be plugged from the QEMU monitor or command-line, and their
properties can be configured as arguments to the <code>-device</code> option or
<code>device_add</code> command.</p>

<p>From the original code:</p>

<blockquote>
  <p>The theory here is that it should be possible to create a machine without
knowledge of specific devices.  Historically board init routines have
passed a bunch of arguments to each device, requiring the board know
exactly which device it is dealing with.  This file provides an abstract
API for device configuration and initialization.  Devices will generally
inherit from a particular bus (e.g. PCI or I2C) rather than
this API directly.</p>
</blockquote>

<p>Some may argue that qdev doesn’t <em>exist</em> anymore, and was
<em>replaced</em> by QOM. Others (including myself) describe it as being
<em>built on top</em> of QOM. Either way you describe it, the same
features provided by the original qdev code are provided by the
QOM-based code living in <code>hw/core</code>.</p>

<p>See also:</p>
<ul>
  <li>KVM Forum 2010 talk by Markus Armbruster: <em>“QEMU’s new device model qdev”</em> (<a href="http://www.linux-kvm.org/images/f/fe/2010-forum-armbru-qdev.pdf">slides</a>)</li>
  <li>KVM Forum 2011 talk by Markus Armbruster: <em>“QEMU’s device model qdev: Where do we go from here?”</em> (<a href="http://www.linux-kvm.org/images/b/bc/2011-forum-armbru-qdev.pdf">slides</a>. <a href="https://youtu.be/Cpt5Zqs_Iq0">video</a>)</li>
  <li>KVM Forum 2013 talk by Andreas Färber: <em>“Modern QEMU Devices”</em> (<a href="http://www.linux-kvm.org/images/0/0b/Kvm-forum-2013-Modern-QEMU-devices.pdf">slides</a>, <a href="https://youtu.be/9LXvZOrHwjw">video</a>)</li>
</ul>

<h2 id="qom">QOM</h2>

<p>QOM is short for <em>QEMU Object Model</em> and was <a href="https://github.com/qemu/qemu/commit/2f28d2ff9dce3c404b36e90e64541a4d48daf0ca">introduced in 2011</a>.
It is heavily documented on <a href="https://github.com/qemu/qemu/blob/master/include/qom/object.h">its header file</a>.
It started as a generalization of qdev. Today the device tree and
backend objects are managed through the QOM object tree.</p>

<p>From its documentation:</p>

<blockquote>
  <p>The QEMU Object Model provides a framework for registering user creatable
types and instantiating objects from those types.  QOM provides the following
features:</p>

  <ul>
    <li>System for dynamically registering types</li>
    <li>Support for single-inheritance of types</li>
    <li>Multiple inheritance of stateless interfaces</li>
  </ul>
</blockquote>

<p>QOM also has a property system for introspection and
object/device configuration. qdev’s property system is built on
top of QOM’s property system.</p>

<p>Some QOM types and their properties are meant to be used
internally only (e.g. some devices that are not pluggable and
only created by machine code; accelerator objects). Some types can
be instantiated and configured directly from the QEMU monitor or
command-line (using, e.g., <code>-device</code>, <code>device_add</code>, <code>-object</code>,
<code>object-add</code>).</p>

<p>See also:</p>
<ul>
  <li>KVM Forum 2014 talk by Paolo Bonzini: <em>“QOM exegesis and apocalypse”</em> (<a href="http://www.linux-kvm.org/images/9/90/Kvmforum14-qom.pdf">slides</a>, <a href="https://youtu.be/fnLJn7PKhyo">video</a>).</li>
</ul>

<h2 id="vmstate">VMState</h2>

<p>VMState was <a href="https://github.com/qemu/qemu/commit/9ed7d6ae0fe7abb444c65caaadb5ef307df82c60">introduced in 2009</a>.
It was added to change the device state saving/loading (for
savevm and migration) from error-prone ad-hoc coding to a table-based approach.</p>

<p>From the original commit:</p>

<blockquote>
  <p>This patch introduces VMState infrastructure, to convert the save/load
functions of devices to a table approach.  This new approach has the
following advantages:</p>
  <ul>
    <li>it is type-safe</li>
    <li>you can’t have load/save functions out of sync</li>
    <li>will allows us to have new interesting commands, like dump <device>, that shows all its internal state.</device></li>
    <li>Just now, the only added type is arrays, but we can add structures.</li>
    <li>Uses old load_state() function for loading old state.</li>
  </ul>
</blockquote>

<p>See also:</p>
<ul>
  <li>KVM Forum 2010 talk by Juan Quintela: <em>“Migration: How to hop from machine to machine without losing state”</em> (<a href="http://www.linux-kvm.org/images/c/c4/2010-forum-migration.pdf">slides</a>)</li>
  <li>KVM Forum 2011 talk by Juan Quintela: <em>“Migration: one year later”</em> (<a href="http://www.linux-kvm.org/images/1/1e/2011-forum-migration.pp.pdf">slides</a>, <a href="https://youtu.be/Mhac35QQWSw">video</a>)</li>
  <li>KVM Forum 2012 talk by Michael Roth: <em>“QIDL: An Embedded Language to Serialize Guest Data Structures for Live Migration”</em> (<a href="http://www.linux-kvm.org/images/b/b5/2012-forum-qidl-talk.pdf">slides</a>)</li>
</ul>

<h2 id="qmp">QMP</h2>

<p>QMP is the <em>QEMU Machine Protocol</em>. <a href="https://github.com/qemu/qemu/commit/9b57c02e3e14163b576ada77ddd1d7b346a6e421">Introduced in 2009</a>. From <a href="https://github.com/qemu/qemu/blob/master/docs/qmp-intro.txt">its documentation</a>:</p>

<blockquote>
  <p>The QEMU Machine Protocol (QMP) allows applications to operate a
QEMU instance.</p>

  <p>QMP is <a href="http://www.json.org">JSON</a> based and features the following:</p>

  <ul>
    <li>Lightweight, text-based, easy to parse data format</li>
    <li>Asynchronous messages support (i.e. events)</li>
    <li>Capabilities Negotiation</li>
  </ul>

  <p>For detailed information on QMP’s usage, please, refer to the following files:</p>

  <ul>
    <li>qmp-spec.txt      QEMU Machine Protocol current specification</li>
    <li>qmp-commands.txt  QMP supported commands (auto-generated at build-time)</li>
    <li>qmp-events.txt    List of available asynchronous events</li>
  </ul>
</blockquote>

<p>See also: KVM Forum 2010 talk by Luiz Capitulino, <a href="http://www.linux-kvm.org/images/1/17/2010-forum-qmp-status-talk.pp.pdf">A Quick Tour of the QEMU Monitor Protocol</a>.</p>

<h2 id="qobject">QObject</h2>

<p>QObject was <a href="https://github.com/qemu/qemu/commit/5a1a2356490399c9b7eb850f9065af554b18cfd1">introduced in 2009</a>.
It was added during the work to add QMP. It provides a generic
<code>QObject</code> data type, and available subtypes include integers,
strings, lists, and dictionaries. It includes reference counting.
It was also called <em>QEMU Object Model</em> when the code was
introduced, but do not confuse it with <em>QOM</em>.</p>

<p>It started a as simple implementation, but was expanded later to support
all the data types defined in the QAPI <em>schema</em> (see below).</p>

<h2 id="qapi">QAPI</h2>

<p>QAPI was introduced in 2011. The original
documentation (which can be outdated) can be seen at
<a href="">http://wiki.qemu.org/Features/QAPI</a>.</p>

<p>From <a href="https://www.mail-archive.com/qemu-devel@nongnu.org/msg55267.html">the original patch series</a>:</p>

<blockquote>
  <p>Goals of QAPI</p>

  <p>1) Make all interfaces consumable in C such that we can use the
   interfaces in QEMU</p>

  <p>2) Make all interfaces exposed through a library using code
   generation from static introspection</p>

  <p>3) Make all interfaces well specified in a formal schema</p>
</blockquote>

<p>From <a href="https://github.com/qemu/qemu/blob/master/docs/qapi-code-gen.txt">the documentation</a>:</p>

<blockquote>
  <p>QAPI is a native C API within QEMU which provides management-level
functionality to internal and external users. For external
users/processes, this interface is made available by a JSON-based wire
format for the QEMU Monitor Protocol (QMP) for controlling qemu, as
well as the QEMU Guest Agent (QGA) for communicating with the guest.
The remainder of this document uses “Client JSON Protocol” when
referring to the wire contents of a QMP or QGA connection.</p>

  <p>To map Client JSON Protocol interfaces to the native C QAPI
implementations, a JSON-based schema is used to define types and
function signatures, and a set of scripts is used to generate types,
signatures, and marshaling/dispatch code. This document will describe
how the schemas, scripts, and resulting code are used.</p>
</blockquote>

<p>See also:</p>
<ul>
  <li>KVM Forum 2011 talk by Anthony Liguori: <em>“Code Generation for Fun and Profit”</em> (<a href="http://www.linux-kvm.org/images/e/e6/2011-forum-qapi-liguori.pdf">slides</a>, <a href="https://youtu.be/YIO34fz8ans">video</a>)</li>
</ul>

<h2 id="visitor-api">Visitor API</h2>

<p>QAPI includes an API to define and use
<a href="https://en.wikipedia.org/wiki/Visitor_pattern">visitors</a> for the
QAPI-defined data types. Visitors are the mechanism used to
serialize QAPI data to/from the external world (e.g. through QMP, the
command-line, or config files).</p>

<p>From <a href="https://github.com/qemu/qemu/blob/master/include/qapi/visitor.h">its documentation</a>:</p>

<blockquote>
  <p>The QAPI schema defines both a set of C data types, and a QMP wire
format.  QAPI objects can contain references to other QAPI objects,
resulting in a directed acyclic graph.  QAPI also generates visitor
functions to walk these graphs.  This file represents the interface
for doing work at each node of a QAPI graph; it can also be used
for a virtual walk, where there is no actual QAPI C struct.</p>

  <p>There are four kinds of visitor classes: input visitors (QObject,
string, and QemuOpts) parse an external representation and build
the corresponding QAPI graph, output visitors (QObject and string) take
a completed QAPI graph and generate an external representation, the
dealloc visitor can take a QAPI graph (possibly partially
constructed) and recursively free its resources, and the clone
visitor performs a deep clone of one QAPI object to another.  While
the dealloc and QObject input/output visitors are general, the string,
QemuOpts, and clone visitors have some implementation limitations;
see the documentation for each visitor for more details on what it
supports.  Also, see visitor-impl.h for the callback contracts
implemented by each visitor, and docs/qapi-code-gen.txt for more
about the QAPI code generator.</p>
</blockquote>

<h2 id="the-end">The End</h2>

<p>Although large, this list is incomplete. In the near future, I
plan to write about QAPI, QOM, and QemuOpts, and how they work
(and sometimes don’t work) together.</p>

<p>Most of the abstractions above are about <em>data modeling</em>, in one
way or another. That’s not a coincidence: one of the things I
want to write about are how some times those data abstractions
have conflicting world views, and the issues resulting from that.</p>]]></content><author><name>Eduardo Habkost</name></author><category term="virt" /><summary type="html"><![CDATA[Having seen many people (including myself) feeling confused about the purpose of some QEMU’s internal APIs when reviewing and contributing code to QEMU, I am trying to document things I learned about them.]]></summary></entry><entry><title type="html">First Post</title><link href="https://habkost.net/posts/2016/11/first-post.html" rel="alternate" type="text/html" title="First Post" /><published>2016-11-22T14:33:16+00:00</published><updated>2016-11-22T14:33:16+00:00</updated><id>https://habkost.net/posts/2016/11/first-post</id><content type="html" xml:base="https://habkost.net/posts/2016/11/first-post.html"><![CDATA[<p>After delaying this for a long time, my new blog is now live.</p>

<p>I plan to use it mostly to write posts about things I have been working on.
Expect content about QEMU, KVM, libvirt, and other virtualization stuff.</p>]]></content><author><name>Eduardo Habkost</name></author><summary type="html"><![CDATA[After delaying this for a long time, my new blog is now live. I plan to use it mostly to write posts about things I have been working on. Expect content about QEMU, KVM, libvirt, and other virtualization stuff.]]></summary></entry></feed>