It’s always good practice to implement the networking part of a project early on, this way you can design the other components around it, thus avoiding some major refactoring in the later stages of development. This article covers how I implemented networking in a planet renderer, and the difficulties I faced during the process. I also talk about road rendering using a vectorial graph system, skeleton based animation using the ozz animation library and finally a particle system using OpenGL’s transform feedback.

Networking:

Network programming is a very challenging field, the idea that the information relayed between a server and a client takes a certain amount of time to reach it’s destination, if it ever reaches it, makes any trivial implementation suddenly much more complex. Most modern games find clever ways to give the player the illusion that everything occurs in real-time. On the server-side we usually try to predict the client actions using techniques such as dead reckoning, while on the client-side, we simultaneously perform the action and send the message to the server, correcting the result based on the given response. These approaches solve most of the issues that can be predicted and corrected, but some systems simply can’t be truly synchronized across a network, the major one being dynamic physics simulation. The reason is actually quite simple, if a desynchronization occurs between the server and a client, it’s impossible to correct the dynamic simulation while it’s still happening, at least as far as I’m aware. It would actually be possible to work around this by performing the simulation on each connected clients, but we are limited by the fact that the results would vary due to some precision issues that usually occurs across different machines (Unless we are using a deterministic physics engine).

Replica System:

In some cases, it really helps to build an architecture around a simple concept, the replica system is built around the idea that a class should be replicated on both the server and the client, and should be synchronized in a continuous way based on a fixed interval. Since I don’t know a whole lot about low-level networking, I used RakNet‘s ReplicaManager3 implementation, it allowed me to start with an already optimized base, meaning I could concentrate more on how to do the actual implementation. The main challenge was to make the core framework side completely versatile, allowing as much freedom on the scripting side as possible. (You can view the client script here, the server script here, the object_base script here and the player script here)

Here’s a simple usage breakdown:

• The server creates a network object, and waits for incoming connections.

class 'server' (sf.Window)

function server:__init(w, h, title, style)
    sf.Window.__init(self, sf.VideoMode(w, h), title, style, sf.ContextSettings(0, 0, 0))

    -- create server network object
    self.network_ = root.Network(root.Network.SERVER_TO_CLIENT)
    self.network_:set_auto_serialize_interval(100)

    -- signal called when network is ready
    root.connect(self.network_.on_connect, function()

        -- we create the shared planet data here
        self.planet_ = planet("sandbox/untitled_planet.json")
    end)

    -- signal called on object creation request
    root.connect(self.network_.on_request, function(bs, network_id)

        local type_id = bs:read_integer()
        local object = nil

        if type_id == ID_PLAYER_OBJECT then

            object = player(self.planet_)
            object:on_network_start(self.network_) -- replica created here
        end

        -- reference object creation here
        return object.replica_
    end)

    -- on_connect is called when ready
    self.network_:connect("localhost", 5000)
end

• The client creates a network object, connects to it, and sends a player creation request to the server.

class 'client' (sf.Window)

function client:__init(w, h, title, style)
    sf.Window.__init(self, sf.VideoMode(w, h), title, style, sf.ContextSettings(0, 0, 0))
    self.active_player_ = nil
    self.player_id_ = 0

    -- create client network object
    self.network_ = root.Network(root.Network.CLIENT_TO_SERVER)

    -- signal called when connected to server
    root.connect(self.network_.on_connect, function()

        -- we create the shared planet data here
        self.planet_ = planet("sandbox/untitled_planet.json", true)

        local bs = root.BitStream()
        bs:write_integer(ID_PLAYER_OBJECT)

        -- send player creation request, return requested network id
        self.player_id_ = self.network_:request(bs)
    end)

    -- signal called on object creation request (after reference from server)
    root.connect(self.network_.on_request, function(bs, network_id)

        local type_id = bs:read_integer()
        local object = nil

        if type_id == ID_PLAYER_OBJECT then

            object = player(self.planet_)
            object:on_network_start(self.network_) -- replica created here

            if self.player_id_ == network_id then
                self.active_player_ = object
            end
        end

        return object.replica_
    end)

    -- on_connect is called when connected to server
    self.network_:connect("localhost", 5000)
end

• The server receives the creation request, creates the object, assigns the requested network id and replicate it across the connected clients. Let’s take a look at what a replica implementation looks like.

class 'object_base_net' (root.NetObject)

function object_base_net:__init(network, object)
    root.NetObject.__init(self, network)
    self.type = root.ScriptObject.Dynamic
    self.object_ = object

    if not network.autoritative then
        self.history_ = root.TransformHistory(10, 1000)
    end
end

function object_base_net:__write_allocation_id(connection, bs)

    root.NetObject.__write_allocation_id(self, connection, bs) -- call first
    bs:write_integer(self.object_.type_id_)
end

function object_base_net:__serialize_construction(bs, connection)

    bs:write_vec3(self.object_.transform_.position)
    bs:write_quat(self.object_.transform_.rotation)
    root.NetObject.__serialize_construction(self, bs, connection)
end

function object_base_net:__deserialize_construction(bs, connection)

    self.object_.transform_.position = bs:read_vec3()
    self.object_.transform_.rotation = bs:read_quat()
    return root.NetObject.__deserialize_construction(self, bs, connection)
end

function object_base_net:__serialize(parameters)

    parameters:get_bitstream(0):write_vec3(self.object_.transform_.position)
    parameters:get_bitstream(0):write_quat(self.object_.transform_.rotation)
    return root.Network.BROADCAST_IDENTICALLY
end

function object_base_net:__deserialize(parameters)

    self.object_.transform_.position = parameters:get_bitstream(0):read_vec3()
    self.object_.transform_.rotation = parameters:get_bitstream(0):read_quat()
    self.history_:write(self.object_.transform_)
end

function object_base_net:__update(dt)

    if not self.network.autoritative then
        self.history_:read(self.object_.transform_, self.network.auto_serialize_interval)
    end
end

• When the object creation request is received on the client-side, we create the object, check if the network id matches the requested one, if it does we assign it as the active player. We now have a replicated object shared across the network.

Vectorial and Residual Data (WIP):

Although it’s possible to generate very realistic and detailed terrain using all sorts of different algorithms, a real life terrain differs a lot from just its own elevation data. It is made out of several kinds of area that each follows their own specific set of rules, a field or a city is generally more flat, a road will flatten the terrain surrounding it, a river will only flow downhill, and so on. In order to accurately represent those elements, we can use the concept of a Graph, holding all this information in the form of vectorial data. Per instance, a simple road can be represented by a curve, linked by two control nodes. In order to apply this road on a terrain, we use the concept of a GraphLayer, since a road affects both the elevation and the color of the terrain, we make use of two different GraphLayer, both are using the same Graph, but are drawn differently. I’m still in the process of understanding how it all works, I will post a more detailed explanation in a future post, for now I only managed to get a simple road with a fixed height working.

Residual data is what allows a the terrain to be deformed, it is the elevation difference between a generated tile and the modified values. It is useful to manually adjust the terrain where needed, but can also be extended to support various types of tools such as a heightmap brush that can print a generated heightmap from other software such as World Machine or L3DT, a noise tool that can apply certain type of noise on top of the generated one, a flatten tool that can equalize the surrounding elevation, and so on. (You can view the planet_producer script here and the road_graph_layers script here)

Skeleton Based Animation (WIP):

I’ve been meaning to implement skeleton based animation for a while now. I used the Assimp library in the past but the loading time where less than ideal for a streaming solution. After looking around for other implementations, I found the ozz-animation library. It’s worth saying that this is not a drop-in solution and will not work out of the box, however once you take the time to write an implementation that fits your needs, it becomes by far the most complete solution out there. It performs the skinning job on the CPU, which is exactly what I need since I already use the GPU quite extensively, and it also allows the process to be done on a seperate thread. Additionally, it comes with a set of command line based tools that will convert a FBX animation to it’s custom binary based archive format, making the loading process very fast. I spent a couple of days learning a basic animation making workflow, and let’s just say I’ve gained a whole lot of respect for all the animators out there. (You can view the player script here)

References:

[1] Source Multiplayer Networking
[2] Real-time rendering and editing of vector-based terrains
[3] Particle System using Transform Feedback