13 KiB
+++ date = "2018-02-04T17:00:00-04:00" title = "Citra Progress Report - 2017 Q4" tags = [ "progress-report" ] author = "anodium" forum = 9777 +++
New year, new changes in Citra. Specifically, lots of technical changes under the hood, from applets to IPC, have made Citra even more accurate and laid a foundation for even more goodies the next year. With many great changes all around, this article is going to be packed to the brim with all the new goodies that have come to Citra during those crimson months.
But, enough faffing about! Let's get right into it:
citra-qt : Adding fullscreen mode by Styleoshin
One of the most requested features for Citra has been enlarging the window to cover the entirety of the screen. After almost a year of requests on both the Citra Discourse and Discord servers, Styleoshin has finally delivered!
{{< figure src="/images/entry/citra-progress-report-2017-q4/fullscreen.png" title="No more pesky window décor!" >}}
Simply go into View → Fullscreen, or just strike the Alt + Return keycombo, and
enjoy Citra to its fullest on your largest monitor.
Rewrite AM Service and Add CIA Installation (Here, here, here, there, this, and that) by shinyquagsire23 and BreadFish64
The am service on a Nintendo 3DS handles the installation, tracking, and removal
of applications and games installed on the console and SD card(s) via a centralized
database in the NAND, with some parts of the database for SD card-installed titles
on the SD card itself. It has a handful of nice features such as installing packages
as they're streamed in, along with DLC support being cleanly implemented as a
side-effect of the CIA format's design. CIAs themselves are also designed to be
streamed in, which is used extensively across 3DS apps, such as the eShop streaming
downloads directly into am so that it installs at the same time, or dlp using
it to stream Download Play CIAs from one 3DS to another directly.
The CIA format starts off with a header that describes where each component of itself starts within the stream and a bitfield. This is then followed by a certificate signed by Nintendo, the CIA's ticket, and the title metadata (or TMD for short). The TMD then contains a list of every CXI file the CIA could possibly have, along with their name, size, etc.
{{< figure src="/images/entry/citra-progress-report-2017-q4/oot3dcia.png" title="A diagram of The Legend of Zelda: Ocarina of Time 3D's CIA" >}}
The astute reader might've noticed I said that the TMD contains a list of every possible CXI. This is because a CIA might not contain every CXI for the game, such as in the case of DLC. The bitfield I mentioned in the header is used to list off which of the entries in the TMD actually exists within that CIA. This is (ab)used in some cases, such as the Home Menu Themes, in that every single theme that exists is actually a different CXI within the same CIA. Just tick the themes the user owns in the bitfield and then attach those CXIs at the end of it. Dead simple way for it to work perfectly with both streaming of the file, and making personalized CIAs for each eShop user.
{{< figure src="/images/entry/citra-progress-report-2017-q4/menucia.png" title="A tiny portion of the Home Menu Themes CIA" >}}
For the am side of streaming installations, the way it handles a request to
install an application is by creating and then giving the requester a handle to
a virtual file. This leaves most of the busy work to am, while the app only needs
to worry about writing the CIA file into that handle.
shinyquagsire23 reimplemented the entirety
of how Citra handles NCCH files, which quickly paved the way to implementing am
in Citra. This allowed users to use FBI to install applications from its virtual
SD card. He and BreadFish64 also went a step
further and added support for installing CIAs via the SDL and Qt frontends
directly, removing the need to use FBI. Do note that you cannot yet run installed
applications and games directly, but CXI executables can access the virtual SD
card for DLC, updates, etc.
Qtifw build installer by j-selby
Citra now has a fancy new installer and updater, thanks to j-selby and jroweboy's efforts! If you haven't already, we strongly recommend you download and install Citra through it, since it will make updating as easy as re-running the installer. You can check it out over here,
macOS: Build x86_64h slice by MerryMage
A little known feature in macOS is fat binaries, the ability to have the same executable contain binaries for multiple architectures. This was used extensively around 2007 to support Mac OS X's transition from PowerPC to x86_64, allowing developers to have one binary work on all Macs effortlessly.
After poking around Apple's LLVM fork, MerryMage found that they had a specific flag to build binaries for Intel Haswell and later. Enabling it made a fat binary that contained both pre-Haswell and post-Haswell code, allowing Citra to take advantage of the newer CPU's instructions, without dropping support for older ones.
Kernel/IPC: Add a small delay after each SyncRequest to prevent thread starvation. by Subv
When communicating with services, titles on a real Nintendo 3DS expect to wait some amount of time before the service replies or responds. Until now, Citra didn't implement this, it responded without incrementing the virtual clock. From the emulated system's perspective, it appeared as though services replied and responded literally instantly because of this, leading to strange side-effects.
For example, some buildings in Animal Crossing: New Leaf fail to render and cannot be interacted with.
{{< figure src="/images/entry/citra-progress-report-2017-q4/acnl-before.png" title="Animal Crossing: New Leaf's acting up" >}}
Or worse, some games such as Star Fox 3D can't even reach the title screen.
{{< figure src="/images/entry/citra-progress-report-2017-q4/fsUSER-threads.png" title="A demonstration of how instant service replies can break a title" >}}
The example in this figure is taken directly from Star Fox 3D. When the game boots, it checks if any save data exists on the console's SD card, and tries to recreate it if it doesn't. It does this by sending a request to the service responsible for file access, and then waiting for a reply. Because the request is synchronous, the thread that made the request is put to sleep and its priority lowered. The game actually uses this time that would otherwise be spent waiting by having other threads run while the original thread was still asleep, so that by the time it woke up with the response, it would have other resources it needed ready to go.
{{< figure src="/images/entry/citra-progress-report-2017-q4/acnl-after.png" title="Animal Crossing: New Leaf, among other games, are now working perfectly with this change!" >}}
As Citra's responses are instantaneous from the point-of-view of the game, the secondary thread doesn't have nearly enough time to finish its job. And, the first thread's priority comes back up above the secondary's once it wakes up, this leads to the first thread waiting on the secondary thread to finish, but it never gets a chance to, due to its now much lower priority. Essentially, the first thread waits on the secondary forever, because the secondary never gets a chance to actually finish what it was doing.
ds84182 and Subv each wrote homebrew software which found and measured the issue, respectively, which B3n30 ran on a real Nintendo 3DS. With which, an average delay for every type of service reply was found. Then, Subv made Citra's virtual clock increment by this amount before fulfilling any service request, solving many of the issues this brought.
core/arm: Improve timing accuracy before service calls in JIT by MerryMage
As our previous section demonstrates, Citra's virtual clock should be incremented before any time code that was supposed to be on a real Nintendo 3DS runs within Citra, including services. This change by MerryMage makes services increment the clock before they are called, instead of after.
One of the reasons this is important is that services can schedule calls to other services to run in the future. If the current time in the virtual clock is incorrect when the service schedule an event, it would run too early since the time for the scheduled event would effectively be shifted back by the virtual time it takes for the service that scheduled it would run.
Although this doesn't fix any reported bugs, it does make the timing emulation of Citra much more accurate.
citra-qt : Fix a bug in our fullscreen implementation by FearlessTobi
In Citra's GUI, a strange bug exists, where if you have fullscreen enabled, and you start a game while the window is maximized, the window instead unmaximizes before starting the game. This was due to Citra not keeping track of the window's state (be it normal, maximized, minimized, or fullscreen), and instead simply defaulting to normal. FearlessTobi found, reported, and fixed the bug. Citra now keeps track of the window's state, position, and size, every time it goes fullscreen and restores them every time it leaves fullscreen.
Allow input configuration with SDL joysticks by muemart
In the input configuration menu, there has been the ability to change the key bindings on the keyboard for years now. Unfortunately, this menu allowed rebinding to keyboard keys only, not gamepads or any other input device one could use. This led to guides such as this one being written, to help users manually change the configuration files if they wanted to use gamepads with Citra.
This was extremely inconvenient and user-unfriendly to most, and so muemart took it upon themselves to finally add support for configuring gamepads within the configuration menu. Now, it's as simple as clicking a button, and pressing the corresponding button on your controller, to set it up.
{{< figure src="/images/entry/citra-progress-report-2017-q4/controller.png" title="Citra's input configuration menu" >}}
shader_jit_x64_compiler: Remove ABI overhead of LG2 and EX2 by MerryMage
The shader JIT in Citra is a component of the video core responsible for recompiling GPU shaders for the 3DS to x86 code, so that they can be run on the user's CPU directly.
Shader instructions like LG2 (calculate binary logarithm) and EX2 (calculate binary exponential) can be run potentially thousands of times per second in a typical Nintendo 3DS title, so it's usually very worthwhile to try to optimize these as best as possible.
In doing so, MerryMage actually rewrote these two instructions to pure x86 assembly, and runs them inline with the rest of the instructions. A faster algorithm, combined with not having to deal with the overhead of calling an external library for math functions, led to this change almost halving the amount of time it takes to calculate these!
{{< figure src="/images/entry/citra-progress-report-2017-q4/shaderjit.png" title="Graph comparing times before and after this change" >}}
Conclusion
As always, changes big or small are vital to the project. Brick by brick, Citra will eventually be an accurate enough emulation of the Nintendo 3DS for most, if not all, uses. Thank you everyone for the hard work you've poured in.
{{< figure src="/images/entry/citra-progress-report-2017-q4/promo.png" title="And more to come." >}}