April Development Update!

Most of my focus is to get Sojour ported to 64 bit and running on a .net 10 platform.

If I pull this off, you get extra performance and I will get access to additional tools that will allow me to do things with Sojour that I have always wanted to do!

The big hold up has been Ionian, Sojour’s custom graphics engine. I wrote it a long time ago for another project of mine called Ancient Armies. It’s not a bad engine, but it is 32 bit and it can only render to one window at a time.

A few posts back I showed off some of my work with Dorian – Ionian’s replacement.

What was probably not made too clear, is that it was a POC:

The original POC! Best viewed on You-Tube to see it in 4k!

What’s a POC?

A POC is a Proof Of Concept. The idea being that for complex tasks – like a graphics engine – you knock up a quick and dirty system purely as a learning exercise. That way when you work on the real thing, you will have a lot more relevant experience under your belt.

The key point of a POC is that you learn what you need to learn, de-risk what needs de-risking and then you effectively throw it away. The lessons learned are then applied to creation of the full enterprise system.

For the last month or so I have been doing just this – working on the real Dorian Engine and not the POC.

The actual engine is way more powerful, expressive and scalable when compared to the POC. It doesn’t look as good – at least not yet – simply because it is at an earlier point in its life cycle.

The biggest difference between Dorian and the POC is that Dorian uses an enterprise grade architecture known as ECS (Entities, Components & Systems).

I think it would be fair to say that ECS takes a bit of getting used to. However, it is massively scalable, simple to use and expressive in ways that are only limited by your own imagination.

I should probably warn you that the next section of this post is going to take the geek factor up to 100! Those of a nervous disposition might well want to bail out about now 😊 (I need to go technical in order to show you the main highlights of Dorian’s ECS system.)

Dorian is now at a point where its architecture is all in place – all I need to do is keep adding features to it – which the new architecture makes exceedingly easy to do.

Here is a short video of me testing three different cameras all pointed at the same world using Dorian. Each window is showing exactly the same thing but from three different perspectives.

This is the real Dorian engine in action. You can see me showing off dynamic window resizing and me trying my best to upset the chase camera, but to no avail! Still work in progress! Best viewed on You-Tube to see it in 4k. (It’s a little dark and flat because YouTube converted a HDR recording to an SDR recording!)

What the video shows are three dynamically resizable windows (Ionian in Sojour can only render to a single window) where each window is pointed at the same world. Dorian allows a lot of flexibility here. You could for example have a few windows rendering world number 1 and another few rendering world number 2 and so forth.

The left hand pane in the video shows the cubes’ actual movement because the camera is not moving. The ones on the right show things from the perspective of the treasure chest cube – one is a follow camera, the other is a first person camera.

In a standard engine like Ionian, coding the animation of all the cubes and the advanced camera behaviours such as the chase camera would have been quite difficult. But, with Dorian’s ECS system it is almost trivial.

What is ECS precisely?

It’s a system where you have Entities and that these entities can represent anything – cameras, models, sound, AI, animations, environments – absolutely anything you can think of.

The key concept of ECS is that they are all treated the same.

The way it works is that the entities are just an Id and that’s it!

Id’s alone don’t do much. However, ECS allows you to dynamically attach (or detach) Components to any entity and these components contain just data. The idea being that if your entity needs to store some specific data, you just attach the appropriate component to it that can store that specific data.

Ok, so you now have entities with data. How is that useful?

Well, the next thing you can do is attach or detach behavioural tags to an entity. Systems within Dorian look out for these tags and operate on the entities that have them using the data stored in the attached components. (I told you we were getting technical!)

This makes the engine very simple to use and incredibly expressive. If you want something to behave in a certain way, just attach the appropriate tag! And that’s it. Very, very simple, but very very powerful.

I don’t normally show my code because it is closed source, but I will make an exception here so that you can see how clean and easy Dorian is to work with – please bear in mind these are very early days!

First up initialisation:

This is how you initialise Dorian – Pretty simple – even for a multi-window, multi-world app!

Dorian is incredibly flexible. Rather than offering a fixed rendering pipeline, it allows the caller to precisely define their own rendering pipeline and they can create as many as they need – though in this example I use the same pipeline for all three windows.

This means that developers can precisely pick what they need to go into their pipeline and they can precisely order it how they want. They can even add their own custom pipeline systems – they are not constrained by what Dorian provides.

It’s flexible in that if I wanted reflections for instance, I could just dynamically add a ReflectionsCubeMapRenderPass and the whole thing would just work!

The other thing Dorian provides that neither Ionian nor the POC offer are simulation (not shown) and animation pipelines. Both of these run independently of the rendering pipelines and are just as configurable:

Setting up an animation pipeline and making camera 2 follow the treasure cube!

The code above sets up an animation pipeline precisely the way I want it.

One of the features I added to it is the ability to process ‘Follow’ behaviour – that is to have one entity follow another. I do this by adding a FollowTarget system to the animation pipeline.

This system will be querying Dorian’s ECS system for the relevant entities and then it will act upon them.

For camera 2, we use the FollowTarget system by attaching a ‘Follow’ component to the camera. I have attached this component to the camera as I want the camera to obtain the additional parameters it needs to allow it to follow another entity (the camera itself is an entity)

Finally, the last line adds a behavioural tag to the camera which invokes the follow system for this camera.

That’s it! That’s all the coder has to do to make any entity in Dorian follow any other entity, no matter how complex the movements. It makes the system incredibly easy to use.

If you want the camera to stop following, just remove the tag. You could even change what it’s following and do all of this dynamically.

The beauty of the system is that each entity can have as many behaviours as you want and that these behaviours can be dynamically attached or detached whenever you want and that the same behaviours can be applied to any entity.

For example, you could pretty much use the same code to make one model follow another…

Here is another example of Dorian’s ECS system in action:

A few lines of code to completely animate a cube!

Here we have another entity, this time the cube you see floating in the demo with the big smiley face.

Once more, we attach a data component to give the entity (the smiley cube) the additional parameters it needs to support animations.

Next, we tag it with the IsAnimatedSinsoidally tag to make it rotate and move. That is all I need to do!

If I want the cube to stop moving I remove the tag.

To top it off, that particular animation is a custom one that I added as a developer using Dorian (as opposed to being the developer coding Dorian).

Dorian allows you to add custom Systems, Tags and Components, so you are not limited by what Dorian can do out of the box.

Dorian is an entity based system, so I could just as easily tag a camera with the same tag as above and have the camera automatically move around like the cube! No additional code required.

It’s this feature that makes Dorian so expressive and so powerful.

Finally, one more code example, but this time it is a structural connection:

Three lines of code to create a camera and turn it into a first person camera.

Dorian allows you to connect (or disconnect) any entity to any other entity! Here it happens to be a camera to a cube model, but that’s just a coincidence.

I could have, for example, connected one of the cubes to another cube! (everything in Dorian is an entity) If that other cube was the parent, the cube we just connected would automatically move along with it. That is the expressive power of Dorian.

Dorian also features its own querying system to allow a developer to easily get at any type of entity that they want.

For example, maybe they want Dorian to return all entities with a Follow component, marked with an IsFollowing behavioural flag and that is in possession of the matrices and vectors to position it in space. This is all you would need to do:

Dorian’s query system in action!

It is an odd architecture to work with until you get used to it.

For example in Ionian and in the POC you had an actual camera class, whereas Dorian doesn’t have cameras. It just has entities that happen to have a camera tag and a world transform component attached to it and that’s it. Together this dynamic composition makes a camera – along with the associated system to operate on its data.

I know this post won’t make a lot sense to most people, but hopefully it gets across two things:

Firstly, I’m still hard at work!

And secondly, that what I’m creating – Dorian – is something of an architectural marvel. It will be the jewel in Sojour’s crown.

Dorian will allow Sojour to have multiple map panes open and visible at the same time. You could even use it to send a ‘players’ map to another monitor when using Sojour as a GM assistant.

The possibilities are endless!

That’s enough from me.

Have Fun!

RobP

Sojour 1.6.0.0 Has Released!

The new filing system!

This release is primarily about making Sojour play nicely with Cloud Drives.

I think it would be fair to say that file loading and saving just got a lot more complicated!

It’s one of those releases where a huge amount of work has gone into it, but if I have done my job correctly, you won’t notice much of a difference between this version and the previous one.

Only 161 files changed and that was just for this story!

I won’t go into too much detail as to how the new filing system works – that’s the video’s job which you can view at the end of this post.

What I will say is that this is the first iteration of this new and complex filing system. It has had around 2 weeks of testing, but alas, the hardware and cloud systems that I have access to is somewhat limited, which might impact the efficacy of the testing.

Before installing this update, I highly recommend that all customers backup their data first.

I’m hoping there won’t be any issues, but given the nature of what has been done, it wouldn’t surprise me.

If you do find issues, please contact me using the email at the front of the manual and we will work together to get it resolved!

Other changes included the incorporation of a new enterprise level logging system:

We now have logging!

Logging got added to facilitate the testing of the new filing system. It’s needed because so much now happens behind the scenes that this is the only reliable way to see what’s going on.

These log files are rolling log files, so they will never grow out of control. Once they reach their maximum size, the oldest entries are removed to create enough space for the new ones.

The above log is the standard log, but the system also outputs a much more detailed developers log for when Sojour is run from Visual Studio.

These logs are now included in Sojour’s exception windows if, heaven forbid, Sojour crashes:

The logs covering the day of the crash are now appended to any exception information!

This should enable me to more easily fault find customer issues.

With the inclusion of the robust filing system and logging, I’m starting to feel that Sojour has crossed that boundary from a bedroom project to a full-on enterprise application!

What else got changed?

There is now a new splash screen:

The new splash screen

This was added because the new filing system is now asynchronous. This results in the user interface loading up its various elements in real-time, which is somewhat disconcerting. The splash screen was added to hide this behaviour.

The rendering loop of Ionian – Sojour’s graphics engine also got re-written to take into account the fact that asynchronous file operations can make Sojour look less busy than it actually is. This can result in the graphics engine unintentionally hogging all the resources as it thinks Sojour is idling.

Various additional bugs were also fixed with the character health bars. These were all extreme corner cases. I only found these issues thanks to the new automated integration tests I wrote for this sub-system:

We now have new automated integration tests!

These tests should help prevent a re-occurrence of the emergency fixes that plagued the 1.5 releases.

The other big fix that got included in this release, was in and around how Sojour remembers its windows positions in a multi-monitor environment. There was a bug that prevented the windows being restored to the correct monitors. This should now be fixed – I say should, because the most monitors I could test this on was just two.

The video below provides a much more in depth explanation with regard to the new filing system. I’d encourage all customers to watch it!

That’s it for this release!

Have Fun!

RobP

Of Splash Screens and Cloud Drives!

A prototype of Sojour’s new splash screen!

I don’t normally like splash screens – but had to reluctantly add one to Sojour as its new disk filing system is now asynchronous.

Without a splash screen you can now see the UI elements populating in real-time as each loading process completes. I think this looks a wee bit unprofessional – hence the splash screen to hide the UI updates.

The older synchronous system – which you folks have – used to hide the UI updates simply because Sojour was blocking the main UI thread whilst performing the loading of the data.

Why the change to Sojour’s loading mechanism?

Cloud Drives.

It’s not just the loading mechanism either, everything file based has been overhauled in Sojour to allow it to play well with Cloud Drives such as One-Drive and Google-Drive.

This has been a huge undertaking, but I am now very pleased with the results.

For those that don’t know, Cloud Drives like to perform random management tasks on your files. Whilst they are doing this, they will lock your files at random. These file locks result in Windows throwing errors if Sojour ever tries to touch them – which can be quite often.

To get around this I have built a new filing system for Sojour which includes a file synchronisation manager to deal with the Cloud Drives various antics.

The code changes are huge:

Only 139 changed code files, but whose counting? 🙂

The new synchronisation system even resulted in me having to redesign Sojour’s graphics engine’s rendering loop – not a trivial task, but more on that when I produce a release video for this functionality.

So where are we with this?

The principal coding is done and the initial testing looks positive. However, these code changes touch and affect everything in Sojour – so there is a lot more testing to do before I can release it.

If we are very lucky we could be looking at a release in two weeks time, maybe earlier. It all depends if the testing discovers anything untoward.

Whilst we wait for this update, I highly recommend that users whose data directories are on a Cloud Drive move them to a local drive using Sojour’s file manager (double click the root node – the Sojour node in the assets browser):

How to move your data folder!

That’s it for this update! Hopefully you will hear from me in the next week or so!

Have Fun!

RobP