Azure Pipelines

What's the Problem?

It was brought to my attention that I did not have enough Azure on my resume to be considered for my dream job. Considering that my experience was exactly what I thought they were looking for, I was still told it wasn't enough. What does that even mean? It has been my experience that has led me to new discoveries, new skills, new abilities, not because of what I had learned or done, but because of what I am capable of learning and doing. But how do I show that?

What am I gonna do about it?

I'm going to do what I always do: Visualize my goal, make a plan, and execute.

First is the easiest: I need to visualize my goal. What is my goal? Easy: To get my dream job. Wait, we already established that... Also, that is a bit too broad. I need something a bit more manageable. What was I lacking for my dream job? Azure experience.

Day 1 - New Mission: Acquire (New/More) Azure Experience

Azure is a lot of things. Where do I start? Where I always start when it comes to forming my plan: understanding the needs of the "user," or in this case, my future employer.

Reflecting back on the position: the role is for a development manager sort of position. This person would be overseeing a team dedicated to tools and implementations of processes and technologies to help streamline development efforts across multiple subsidiaries.

Great, I already have experience in creating backlogs and managing projects in Azure (it is how I manage all my around-the-house projects). So I started creating a backlog of items that I figured I'd need to complete. I decided I wanted to cater this to the audience, and figured what better way than creating a game in Unity and establishing a CI/CD pipeline for it?

I started drafting all kinds of requirements for what I felt I needed to build just so I could have things to promote. After which establishing a framework, I could build my pipeline. I estimated this would take 3 sprints, or 6 weeks. My backlog started to look a little something like this:

Yikes! That is a whole lot of stuff. Like... a bunch of stuff... and I haven't even gotten to the CI/CD pipeline yet?

Granted, this was all the way back on Thursday, September 9th, 2021 (today is Tuesday, September 14th), so I didn't know any better yet about what all possibilities I had in front of me.

Now, to the first thing on the list... Game engine... My choice? Unity. I haven't played with Unity in a long time. Like... a long time... but I fiddle with Daz3D and have used the bridge utility with Blender, and wanted to try importing my characters into Unity anyway, so why not give it a download?

So I go and start downloading Unity on my PC.

While it is downloading, I need to do 2 things: 1.check out new features in Unity since the last time I muddled with it, and 2. research interoperability of Unity with Azure.

And there you have it, my nightly reading and YouTube rabbit hole diving has an agenda and a direction.

Day 2 - A New Plan...?
Or The Same Plan... But With A Twist...
No...  That Would Make It A Twisted Plan...
Let's Just Say I Found A New Direction

Okay, subheading the length of modern anime and manga titles aside, I came up with a lot of new information last night.

For starters, Unity has apparently implemented "microgames" since my last endeavor, so I can scrap the need to build anything from scratch as a starting point. It's already accomplished that portion. Besides, this exercise shouldn't be focused on an exhaustive list of game components. I merely need the game components to move around within an Azure pipeline. Bonus points to Unity for this, because now I don't have to waste time on less pertinent features and can basically eliminate 1-2 sprints worth of development!

I was able to discard 90% of my backlog and reduce my work to the following steps:

1. Pool local agent for Azure job

2. Clean up any previous build artifacts

3. Retrieve Library files from previous build (if it exists)

4. Perform Unity build

5. Create standalone installer

6. Publish the build

7. Upload published build to cloud storage

Now the fun can begin.

Day 2 Part 2 - Let the Games Begin!

First thing's first. I need to make sure my environment is fully up to the task... and after perusing Amazon to purchase some new RAM, I decided it was time to create my project. I open Unity Hub and launch the FPS microgame. I needed a codebase to begin with, so this seemed the logical place to begin. Also, with the FPS microgame loaded up, it was a good opportunity to play on the first few tutorials and refamiliarize myself with Unity, which, for some reason, really felt a lot more comfortable to me than I remembered.

After saving the game project and corresponding files, I remembered... this needs to be in my Azure repo... but I like to keep all my Azure repos on my mapped drive hosted on my Synology NAS... Oops...

Okay, well that was fun... this should be simple, right? I should just need to copy my project folder over to my NAS... wait... let me first make sure my repo is properly linked to Azure... I'll start there...

So here were my new series of steps:

1. Create repo in Azure

2. Use git clone to "pull" my empty repo to my "local" drive (mapped drive from NAS)

   • I do this mainly to ensure all my connection and configuration info is correct and my .git folder is properly generated on my local machine

3. I copypasta the Unity project from my local machine to the repo folder

4. Now I just do a git commit followed by a git push and my files are in my repo! w00t w00t!

Or so I thought...

If it seems too easy now, just wait... it'll be frustratingly difficult soon enough...

You know when things seem to be going well and you just think to yourself "what could go wrong?" 

It is usually at that very moment, the universe says "hold my beer."

So once I know that my code is checking in, I can see it in my repo, and it all feels like I can start building out my pipeline, it dawns on me... I have never launched a Unity project from an Azure repo-linked folder... or an existing project from the Unity Hub, for that matter... and it would probably be something I might consider doing before moving too far forward. The last thing I want to do is show this off and have a project and have it crash on me like I'm Bill Gates showing off the latest version of Windows to a TechCrunch audience. So yeah, let's test the game from the newly copied project files.

1. Open Unity Hub

2. Click Add

3. Browse to repo project folder

4. Click Select Folder button

5. Choose the project from the Unity Hub list to launch it

6. .

7. ..

8. ...

9. Somewhere in here the game launches, right?

10. RIIIIIIGHT????

11. Check Task Manager

12. Locates process: Unity.exe

13. Sees status... "Not responding"

14. Begins teaching my children the Americanized version of very specific "French" words

15. New research topic: Moving Unity project folder to new location... "something something Enable Visible Meta Files... check that out... cool, already set... something something if that fails, delete Library and Temp folders..."

16. Okay, so I delete the Temp and Library folders from the project root in my repo directory...

17. Repeat steps 1-13... yes, thru step thirteen... that one that says "Not responding"

18. Further enhance children's foreign vocabulary which institutions refuse to teach in traditional schools

19. Try it again... and yes, this goes thru step 13 once more...

20. New observation time: watch the files to see what is failing to load...

21. It's at this point I notice the timestamps are still updating, even though the process is showing in Task Manager as "Not responding" so I decide to wait... for 30 minutes...

22. Wouldn't you know it? It decided to work! It just needed to get some lunch, I guess

Okay, so after 22 steps later (46 if you count the 13 repeated steps both times they were repeated), I'm looking at what is supposed to be my project loaded up in Unity.

At first, I'm excited because, well, Unity loaded up! and I can see the Assets, Textures, etc. in my editor! But the scene I'm looking at is not at all what I had saved in my original project! In fact, it was pretty empty... So a little panic sets in as I feel like I've wasted an entire day troubleshooting my newly created, and then moved, Unity project and it still isn't working right! But honestly, I have only been troubleshooting this for the past hour and that was the first hour after lunch, so I take a breath and start to study the editor once more... I start poking through the files in the editor and discover that I just didn't have the correct startup scene. I changed to the main theme from the microgame template, and... Voila! I am now teaching French that is classroom appropriate! Erm... I mean... not that "voila" isn't appropriate, but my game loaded from the repo directory!

>git add --all
>git commit --m "Known working project files"
>git push

Whew! Awesome... now for what I was really looking to do... Since I burned so much time on that ordeal, I decided that I would simply work through a subset of the 7 steps I compiled from the previous night's research.

1. Create local agent

2. Clean previous build artifacts

3. Retrieve Library files from previous build (if exists)

4. Perform Unity Build

5. Publish build to local file system

This process went smoother than expected... like the no-issues kind of smooth.

So smooth, in fact, that I decided to implement a portion between steps 4 and 5, to utilize Inno Setup to create a single-file installer for my game, so that anyone could download the single EXE and install the entire game and all dependencies, completely hassle-free.

I had to make some modifications to the config files, which was fun, because I got to dig into the generated files from Inno's wizard and I love digging in and seeing what I need to change to make it work for me.

I got Inno Setup configured, working, and implemented into my project, as well as my pipeline, and was able to get the pipeline to invoke my local job agent, perform steps 1-4, create a standalone installer for the Unity build artifacts, then publish the installer to the local file system, all within 3 hours. All being triggered from updates to the master branch in my repo.

This seemed like a good place to call it quits for the day, and the weekend. I was well ahead of schedule and wanted to be super fresh to tackle the next task.

Day 3 - The Road Less Traveled Or:
Why I Upgrade Suspension and Brakes Before the Engine Or:
How I Learned to Stop Worrying and Love the Bomb

Most of what I've done to this point has shown some level of capability, but at the same time, not much of it is out of the ordinary for what you can find with ready-made tutorials or walkthroughs. Granted, I studied it over one night, and only referred to my notes the next day, or consulting new references specific to the isolated issues as they arose, but I wanted to be sure I was treading uncommon territory. After all, I'm trying to demonstrate my capabilities. I need to create adversity. Or, to reference this section's heading, I need to make sure I can maintain progress under the stress of new conditions.

This is why I decided to save the next step for last. I could go the easy, well-documented route of uploading my standalone installer to an Azure blob and call it a day... but that seems a bit too boring. Also, being a bit of an indie enthusiast, I know more of us prefer cloud hosting services like MEGA. See where I'm going with this?

MEGA bytes... and when it does, it hurts

Good: My automated build process only takes around 3 to 3.5 minutes to execute. Bad: I'm about to be executing it a lot... like alot a lot... like we could knight my build process as Sir Lotsalot... if it was a pet, it would be an axolotl... like I should have named my local job agent "Forrest" because... again... lots of running... like a game of soccer but with less feigned injuries and more red cards... like I could keep going with metaphors and puns, but it still wouldn't be as many as the times this process is going to be executed. Seriously, since everyone is now so well versed in the language, there will be more executions than in the French Revolution... okay not that many... and okay, okay, I'm done...

Cue the new problem statement and queue the curse words:

Since I plan to reuse this automated build process on future projects and be adoptable by indie developers, my build process should utilize MEGA and not be tied to an Azure storage solution.

Not sure why I didn't update this a year ago when it was finished, but...

Long story short, I read lots and lots of stuff online that didn't get me any closer to my goal of integrating Azure directly with MEGA...

But, since I was running on a local build agent, I decided to download, install, and run MEGAcmd which can be found here:
https://mega.io/cmd

Then I setup MEGA-WEBDAV as described in these instructions:
https://github.com/meganz/MEGAcmd/blob/master/contrib/docs/WEBDAV.md

After that, I just setup an additional step in my build pipeline to perform a "PublishBuildArtifacts@1" task to publish a copy of the build to the mapped WEBDAV location.

As long as MEGAcmd is running on the build machine, and the WEBDAV mappings are correctly setup, this works like a charm!

All in all, this took me only a few hours to set it up, make the updates, and run through a few tests to ensure it was all working correctly.