Server page

templates/base.html

@@ -27,6 +27,7 @@
             <td><a href="{{ROOT}}/transport.html">Transport layer</a></td>
             <td><a href="{{ROOT}}/packet.html">Packet format</a></td>
             <td><a href="{{ROOT}}/protocol.html">Application Protocol</a></td>
+            <td><a href="{{ROOT}}/server.html">Let's write a server</a></td>
         </tr>
     </table>
     {% block body %}{% endblock %}

templates/pages/index.html

@@ -8,10 +8,12 @@
     going to have to reverse engineer an open source project (or a closed source one, for that matter), I might as
     well just go reverse engineer an actual game (or it's stdlib, as most of my time has been spent currently).</p>
 <p>For the sake of being lazy, I'll probably end up calling it eAmuse more than anything else throughout these
-    pages. Other names you may come across include <code>httac</code> and <code>xrpc</code>. The latter are the
+    pages. Other names you may come across include <code>httpac</code>* and <code>xrpc</code>. The latter are the
     suite of HTTP functions used in the Bemani stdlib, and the name of their communication protocol they implement
     at the application layer, but whenever someone refers to any of them in the context of a rhythm game, they will
-    be referring to the things documented here.</p>
+    be referring to the things documented here.<br />
+    <small style="margin-left: 8px">*I believe <code>httpac</code> is the official name for the protocol internally.</small>
+</p>
 <p>These pages are very much a work in progress, and are being written <i>as</i> I reverse engineer parts of the
     protocol. I've been asserting all my assumptions by writing my own implementation as I go, however it currently
     isn't sharable quality code and, more importantly, the purpose of these pages is to make implementation of one's
@@ -52,6 +54,12 @@
     <ul>
         <li>There are a crazy number of sub pages here, so just go check the contents there.</li>
     </ul>
+    <li><a href="./server.html">Let's write a server</a></li>
+    <ol>
+        <li><a href="./server.html#groundwork">Groundwork</a></li>
+        <li><a href="./server.html#handlers">Implementing handlers</a></li>
+        <li><a href="./server.html#extra">Extra endpoints</a></li>
+    </ol>
     <li>Misc pages</li>
     <ol>
         <li><a href="./cardid.html">Parsing and converting card IDs</a></li>

templates/pages/protocol.html

@@ -6,11 +6,10 @@
     getting data around. This means we need an HTTP server running but, as we'll see, we don't need to think too
     hard about that.</p>
 <p>Every request made is a <code>POST</code> request, to <code>//&lt;model&gt;/&lt;module&gt;/&lt;method&gt;</code>,
-    with its body being encoded data as described in the previous sections. In addition to the
-    <code>X-Compress:</code> and <code>X-Eamuse-Info:</code> headers previously detailed, there is also a
-    <code>X-PCB-ID:</code> header. that can be set. Your machine's PCB ID uniquely defines the physical board. This
-    header is added in out-bound requests, and allows the server to identify you. Importantly, it's also the value
-    that the server uses to identify which machines are authorized to be on the network, and which are not.
+    with its body being encoded data as described in the previous sections. This behaviour can be altered using the
+    <code>url_slash</code> flag in <code>ea3-config.xml</code>. Disabling this switches to using
+    <code>/?model=...&module=...&method=...</code> for requests instead. Make sure to implement both of these if
+    implementing a server!
 </p>
 <p>Every request is followed immediately by a response. Any response code other than <code>200</code> is considered
     a failure.</p>

templates/pages/server.html

@@ -1,3 +1,286 @@
 {% extends "base.html" %}
 {% block body %}
-{% endblock %}
+<h1>Let's write an e-Amusement server!</h1>
+<p>No, seriously. It's quite easy.</p>
+<p>Before we start anything, let's figure out exactly what we <i>need</i> to implement in order to get games to start.
+    As it turns out, very little.</p>
+<ul>
+    <li><code><a href="{{ROOT}}/proto/services.html#get">services.get</a></code></li>
+    <li><code><a href="{{ROOT}}/proto/pcbtracker.html#alive">pcbtracker.alive</a></code></li>
+    <li><code><a href="{{ROOT}}/proto/message.html#get">message.get</a></code></li>
+    <li><code><a href="{{ROOT}}/proto/facility.html#get">facility.get</a></code></li>
+</ul>
+<p>To make matters even easier, none of these endpoints require any functioning logic! It should be noted that to follow
+    along, however, you will need a functioning packet encoder and decoder.</p>
+<h2 id="groundwork">Groundwork</h2>
+<p>Before we get started, there are a few things we need to get out of the way. One potential elephant in the room is
+    how we tell games to use our server. You may have configured this thousands of times, or maybe this is your first
+    time. Head on over to <code>prop/ea3-config.xml</code>, and edit <code>ea3/network/services</code> to
+    <code>http://localhost:5000</code> (or whatever you want :P). If you can't find it, search for
+    <code>https://eamuse.konami.fun/service/services/services/</code> and swap that out (yes, they really felt the
+    need to repeat service 3 times).
+</p>
+<p>While we're in this file, we need to turn off a few services (for now). This is part of how we're able to start the
+    game with such a minimal server. Right at the bottom of the file there should be a <code>option</code> and
+    <code>service</code> block. Within these we want to turn off <code>pcbevent</code> and <code>package</code>. Totally
+    turning of e-Amusement will usually lead to the game refusing to start, and that's no fun anyway.
+</p>
+<p>We will turn these two back on later, but for now we want everything turned off. (<code>cardmng</code> and
+    <code>userdata</code> aren't used during statup, so don't matter.)
+</p>
+<h3 id="stub-code">Basic code framework</h3>
+<p>I'm going to assume you already have a working packet processor. I have used an intentionally simple API for mine, so
+    hopefully it should be easy to follow along with code samples. In addition to that, to create a server we will need
+    a, well, server. I'm going to be using <code>flask</code>, because I'm using Python, but I'm going to minimise how
+    much flask-specific code I write, so this should really be applicable to any server. With that said, shall we
+    starting writing code?</p>
+
+<pre>{% highlight "python" %}
+from flask import Flask, request, make_response
+
+app = Flask(__name__)
+
+def handle(model, module, method):
+    ea_info = request.headers.get("x-eamuse-info")
+    compression = request.headers.get("x-compress")
+    compressed = compression == "lz77"
+
+    payload = b""  # TODO: This
+
+    response = make_response(payload, 200)
+    if ea_info:
+        response.headers["X-Eamuse-Info"] = ea_info
+    response.headers["X-Compress"] = "lz77" if compressed else "none"
+
+    return response
+
+@app.route("//<model>/<module>/<method>", methods=["POST"])
+def call(model, module, method):
+    return handle(model, module, method)
+
+@app.route("/", methods=["POST"])
+def index():
+    return handle(request.args.get("model"),request.args.get("module"), request.args.get("method"))
+
+if __name__ == "__main__":
+    app.run(debug=True)
+{% endhighlight %}</pre>
+<p>This is all of the flask-specific code I'm going to be writing. It should be fairly simple to follow what it going on
+    here. From within <code>handle</code> we need to:</p>
+<ol>
+    <li>Unpack the request</li>
+    <li>Identify the handler for that method</li>
+    <li>Call the handler</li>
+    <li>Construction and pack the response</li>
+</ol>
+<p>For me, that looks something like:</p>
+<pre>{% highlight "python" %}
+from utils.decoder import decode, unwrap
+from utils.encoder import encode, wrap
+from utils.node import create_root
+
+methods = {}
+# Populate methods
+
+# Step 1.
+call, encoding = decode(unwrap(request.data, ea_info, compressed))
+# Step 2.
+handler = methods[(module, method)]
+# Step 3.
+root = create_root("response")
+handler(call, root)
+# Step 4.
+payload = wrap(encode(root, encoding), ea_info, compressed)
+{% endhighlight %}</pre>
+<p>At this point, you should be able to start the game and see a single request come in for the services method. This
+    endpoint is mandatory for anything else to happen, but if you're able to inspect that one request then you're on the
+    right track.</p>
+
+<h2 id="handlers">Implementing handlers</h2>
+<p>Now that the groundwork is in place, implementing handlers themselves should be a fairly easy task. The first handler
+    we need to implement is <code><a href="{{ROOT}}/proto/services.html#get">services.get</a></code>. You may have
+    noticed in the previous section, but this request is made <i>before</i> the network check is performed. Weird, but
+    okay. Referencing the spec, the response to this method should be a list of every service we support. Luckilly for
+    us, that's not very many right now. My code for this is as follows:</p>
+<pre>{% highlight "python" %}
+from utils.node import append_child
+
+SERVICES_MODE = "operation"
+SERVICE_URL = "http://localhost:5000"
+SERVICES = {
+    "facility": SERVICE_URL,
+    "message": SERVICE_URL,
+    "pcbtracker": SERVICE_URL,
+}
+@handler("services", "get")
+def services_get(call, resp):
+    services = append_child(resp, "services", expire="10800", mode=SERVICES_MODE, status="0")
+    for service in SERVICES:
+        append_child(services, "item", name=service, url=SERVICES[service])
+{% endhighlight %}</pre>
+<p><code>@handler</code> is a helper function I have defined that registers the function into the <code>methods</code>
+    dictionary.</p>
+
+<p>Next on the menu is <code><a href="{{ROOT}}/proto/pcbtracker.html#alive">pcbtracker.alive</a></code>. If we were
+    implementing a full server, handling this would involve looking up the machine in our database, confirming if paseli
+    is allowed, and processing the request accordingly. Luckily for us, that's not what we're doing. We're going to just
+    echo back the enabled flag the machine operator has set.</p>
+
+<pre>{% highlight "python" %}
+@handler("pcbtracker", "alive")
+def pcbtracker(call, resp):
+    ecflag = call[0].ecflag
+
+    append_child(
+        resp, "pcbtracker",
+        status="0", expire="1200",
+        ecenable=ecflag, eclimit="0", limit="0",
+        time=str(round(time.time()))
+    )
+{% endhighlight %}</pre>
+<p>Feel free to pause right now and implement a less trusting solution here. I just didn't particularly feel like it,
+    and the objective of this page is to get a bare-bones server running.</p>
+
+<p>Our next method is <i>even</i> simpler. Again, we <i>should</i> be performing database queries to determine if there
+    are any new messages to send, but we don't, and there won't be!</p>
+
+<pre>{% highlight "python" %}
+@handler("message", "get")
+def message(call, resp):
+    append_child(resp, "message", expire="300", status="0")
+{% endhighlight %}</pre>
+
+<p>Take a breather at this point. I'm really sorry, but the last endpoint we need to imeplement is
+    <code><a href="{{ROOT}}/proto/facility.html#get">facility.get</a></code>. This endpoint is neither simple not small.
+    Well... Okay. Let's cheat. Same deal as ever. We should be looking up all this information (in this instance, we
+    need to check the details about the physical arcade the machine is registered within) but we can hardcode it all.
+    Does much of this data make any sense? Nope. Does it actually get validated by the game? Not really.
+</p>
+<pre>{% highlight "python" %}
+@handler("facility", "get")
+def facility_get(call, resp):
+    facility = append_child(resp, "facility", status="0")
+    location = append_child(facility, "location")
+    append_child(location, "id", Type.Str, "")
+    append_child(location, "country", Type.Str, "UK")
+    append_child(location, "region", Type.Str, "")
+    append_child(location, "name", Type.Str, "Hello Flask")
+    append_child(location, "type", Type.U8, 0)
+    append_child(location, "countryname", Type.Str, "UK-c")
+    append_child(location, "countryjname", Type.Str, "")
+    append_child(location, "regionname", Type.Str, "UK-r")
+    append_child(location, "regionjname", Type.Str, "")
+    append_child(location, "customercode", Type.Str, "")
+    append_child(location, "companycode", Type.Str, "")
+    append_child(location, "latitude", Type.S32, 0)
+    append_child(location, "longitude", Type.S32, 0)
+    append_child(location, "accuracy", Type.U8, 0)
+
+    line = append_child(facility, "line")
+    append_child(line, "id", Type.Str, "")
+    append_child(line, "class", Type.U8, 0)
+
+    portfw = append_child(facility, "portfw")
+    append_child(portfw, "globalip", Type.IPv4, map(int, request.remote_addr.split(".")))
+    append_child(portfw, "globalport", Type.S16, request.environ.get('REMOTE_PORT'))
+    append_child(portfw, "privateport", Type.S16, request.environ.get('REMOTE_PORT'))
+
+    public = append_child(facility, "public")
+    append_child(public, "flag", Type.U8, 1)
+    append_child(public, "name", Type.Str, "")
+    append_child(public, "latitude", Type.S32, 0)
+    append_child(public, "longitude", Type.S32, 0)
+
+    share = append_child(facility, "share")
+    eacoin = append_child(share, "eacoin")
+    append_child(eacoin, "notchamount", Type.S32, 0)
+    append_child(eacoin, "notchcount", Type.S32, 0)
+    append_child(eacoin, "supplylimit", Type.S32, 100000)
+    url = append_child(share, "url")
+    append_child(url, "eapass", Type.Str, "www.ea-pass.konami.net")
+    append_child(url, "arcadefan", Type.Str, "www.konami.jp/am")
+    append_child(url, "konaminetdx", Type.Str, "http://am.573.jp")
+    append_child(url, "konamiid", Type.Str, "http://id.konami.jp")
+    append_child(url, "eagate", Type.Str, "http://eagate.573.jp")
+{% endhighlight %}</pre>
+
+<h2 id="start">Start the game!</h2>
+<p>Go for it, you've earned it.</p>
+<p>If you've done everything right, you should now be able to pass the network check during startup. If you get really
+    lucky, you might be able to insert coins... Yeah okay unfortunately we aren't <i>quite</i> done. It's quite
+    satisfying though getting to the title screen at least, right?</p>
+<p>To unblock the coin mechanism we're going to want to enable the <code>pcbevent</code> option within
+    <code>ea3-config.xml</code>. Don't forget to also update your services endpoint to return a URL for
+    <code>pcbevent</code>. The handler is super simple, at least. (As ever, this should be doing database stuff--logging
+    in this case--but we're not bothering with that.)
+</p>
+
+<pre>{% highlight "python" %}
+@handler("pcbevent", "put")
+def pcbevent(call, resp):
+    append_child(resp, "pcbevent", status="0")
+{% endhighlight %}</pre>
+
+<p>For real, this time, we can start the game.</p>
+
+<figure>
+    <img width="256" src="./images/game_started.png">
+    <figcaption>It lives!</figcaption>
+</figure>
+
+<h2 id="extra">Extra endpoints</h2>
+<p>Remember how we also disabled <code>package</code>? We can go and enable that one too if we want. Assuming you don't
+    plan to offer OTA updates from your server, this endpoint ends up super simple too; just report nothing to download.
+</p>
+
+<pre>{% highlight "python" %}
+@handler("package", "list")
+def package_list(call, resp):
+    append_child(resp, "package", expire="600", status="0")
+{% endhighlight %}</pre>
+
+<h3 id="cardmng">Stub cardmng implementation</h3>
+<p>As with other endpoints, we can get a "working" implementation of e-Amusement cards by returning some generic
+    hardcoded values. Check the reference if you want to properly implement these endpoints, because they aren't
+    terribly complex.</p>
+<pre>{% highlight "python" %}
+cardmng = handler("cardmng")
+@cardmng("inquire")
+def inquire(call, resp):
+    append_child(resp, "cardmng", binded="1", dataid="0000000000000000",
+        exflag="1", expired="0", newflag="0", refid="0000000000000000", status="0")
+
+@cardmng("authpass")
+def authpass(call, resp):
+    append_child(resp, "cardmng", status="0")
+{% endhighlight %}</pre>
+
+<h3 id="sdvx4">Stub SDVX 4 implementation</h3>
+<p>Odds are implementing the <code>cardmng</code> endpoints got you past the card check, but then immediately into a
+    network error, as the game attempted to retrieve your game-specific profile. While I don't know the endpoints for
+    all games, I do know that SDVX 4's can be stubbed out quite simply (below). It should be noted that this works by
+    always returning "player is a new user" in the <code>sv4_load</code> handler, meaning we haven't really achieved
+    much here besides adding an bunch of extra steps players need to take before they can play the game.</p>
+
+<pre>{% highlight "python" %}
+game = handler("game")
+@game("sv4_load")
+def sv4_load(call, resp):
+    game = append_child(resp, "game", status="0")
+    append_child(game, "result", Type.U8, 1)
+@game("sv4_load_m")
+def sv4_load(call, resp):
+    game = append_child(resp, "game", status="0")
+    append_child(game, "music")
+@game("sv4_load_r")
+def sv4_load(call, resp):
+    append_child(resp, "game", status="0")
+@game("sv4_frozen")
+def sv4_load(call, resp):
+    append_child(resp, "game", status="0")
+@game("sv4_new")
+def sv4_load(call, resp):
+    append_child(resp, "game", status="0")
+{% endhighlight %}</pre>
+
+{% endblock %}