Ever needed to bulk reset your upvote history for a subreddit? Yes, it is a bit of a random topic. However, I found myself in this situation recently and there wasn't a way to do so using Reddit's built in capabilities. A simple piece of Javascript came to the rescue!
Disclaimer: This solution is coupled to the markup of the current Reddit website. So will only be functional until Reddit changes their markup in the future.
Start by navigating your browser to: https://www.reddit.com/user/dionbeetson/upvoted/
Then open up your developer console (F12). Click on the console tab.
The javascript code
/**
* Function to clear votes for a specific subreddit
*/
let clearVote = (subReddit) => {
let clearedCounter = 0;
let anchorsFound = document.querySelectorAll("a[href$='/r/" + subReddit + "/']"); for (var i = 0, l = anchorsFound.length; i < l; i++) {
let anchor = anchorsFound[i];
let post = anchor.closest('.Post');
let button = post.querySelector("button[data-click-id='upvote']");
button.click();
clearedCounter++;
}
console.log('Cleared ' + clearedCounter + ' upvotes from ' + subReddit); };
// We need to continually keep scrolling to the bottom to load more history
let scrollToBottomCounter = 0;
let interval = setInterval(() => {
if( scrollToBottomCounter > 300 ) {
clearInterval(interval);
clearVote('Bitcoin');
}
window.scrollTo(0,document.body.scrollHeight);
scrollToBottomCounter++;
}, 1500);
// Call clear vote for the subreddits you want to clear your upvotes for
// EG: this is the name of the subreddit that you can see within the UI that looks like r/CryptoMarkets/
// However you simply include the subreddit name and not the starting 'r/' and trailing '/'
clearVote('InstantRegret');
clearVote('RoastMe');
// You can call clearVote as many times as you require.
How to run the script
Now, after adding a 'clearVote' function call for each subreddit you want to clear upvotes for you can paste this script into the 'console' within your browser and the script will simulate a user clicking the upvote button for all of your upvote history.
Teach you how to code a basic NeuroEvolution implementation
Improve the performance of the NeuroEvolution
Implement the ability to save/load models
In this last tutorial we will:
Learn how to use simple shape/object/color detection to gather the inputs required to feed into TensorFlow JS (instead of interacting directly with the GameAPI object)
We need to create a class that will be used instead of GameAPI to gather all of the required inputs from the game every 10ms and then pass them to TensorFlow JS to make the same prediction to jump or not.
Disclaimer:
There is already logic in js/ai/Ai.js to initialise this class if you select the checkbox 'Use Object Recognition' in the UI
There are a few inputs small items we are not decoupling from the original GameAPI (eg: getScore()), as it was outside the scope of my experiment. But it could be implemented if we really wanted to.
How it works
Every 10ms the game will execute a method to
Extract an image from the game canvas
Convert the image to greyscale
Using a 10x10 pixel grid create a layout of the level in regards to:
What is whitespace (color=white)
What are obstacles to jump over (color=grey)
Where the player is (color=black)
Look ahead 4 blocks (40 pixels) and determine if there is a block or dip to jump over
Determine the x/y coordinates that can feed into TensorFlowJS
Ai.js will then use the same logic as the previous tutorials to determine to jump or not
Create a file called js/ai/GameImageRecognition.js and then paste in the below code.
js/ai/GameImageRecognition.js
class GameImageRecognition {
}
Lets now create the start() method which is called by default in Ai.js which will start the game, setup the canvas tracker (which is essentially a new hidden canvas DOM element that we paint an image of the game canvas into every 10ms) and use it for detecting shapes/objects/player position etc...
js/ai/GameImageRecognition.js
start() {
const self = this;
this.#enableVision = document.querySelector("#ml-enable-vision").checked;
// @todo - remove dependency on gameAPI object - although outside of scope of this example
this.#gameApi.start();
this.setupCanvasTracker();
// Simulate what happens in the game
setTimeout(() => {
self.#isSetup = true;
}, 100);
}
We will also add in a few other helper functions to get things moving.
And add in the required setters for useImageRecognition
js/ai/NeuroEvolution.js
set useImageRecognition( useImageRecognition ) {
this.#useImageRecognition = useImageRecognition;
}
Reload your browser (http://127.0.0.1:8080/), check 'Use Object Recognition' and click 'Start evolution' button. You should see the games begin, but get a lot of game.gameApi.getPlayerY is not a function errors. This is because we need to implement a range of functions to gather input.
Before we do that though, we will add in the logic to extract information from the game canvas every 10ms.
js/ai/GameImageRecognition.js
// Method to extract data from canvas/image and convert it into a readable format for this class to use
extractVisualTrackingData(){
let data = this.#gameApiCanvas.getContext('2d').getImageData(0, 0, this.#visualTrackingCanvas.width, this.#visualTrackingCanvas.height);
let dataGrey = this.convertImageToGreyScale(data);
this.#visualTrackingMap = this.generateVisualTrackingMap(dataGrey, this.#visualTrackingCanvas.width, this.#visualTrackingCanvas.height, this.#visualTrackingMapSize, this.#colors);
this.updatePlayerPositionFromVisualTrackingMap(this.#visualTrackingMap, this.#colors);
this.#sectionAhead = this.getSectionAhead(this.#playerX, this.#playerY, 4, this.#visualTrackingMapSize, this.#playerGroundY);
}
// Method to create an object indexed by xposition and yposition with the color as the value, eg: 10x40 = grey
generateVisualTrackingMap(data, width, height, visualTrackingMapSize, colors) {
let visualTrackingMap = {};
for( let y = 0; y < height; y+=visualTrackingMapSize ) {
for( let x = 0; x < width; x+=visualTrackingMapSize ) {
let col = this.getRGBAFromImageByXY(data, x+5, y+5)
let key = x+'x'+y;
visualTrackingMap[key] = colors.background;
if ( 0 == col[0] ) {
visualTrackingMap[key] = colors.player;
}
if ( col[0] > 210 && col[0] < 240 ) {
visualTrackingMap[key] = colors.block;
}
}
}
return visualTrackingMap;
}
These above functions have extra dependencies. Let's add in functionality to convert an image into greyscale, as well as get the RGBA from a specific pixel on that image.
js/ai/GameImageRecognition.js
convertImageToGreyScale(image) {
let greyImage = new ImageData(image.width, image.height);
const channels = image.data.length / 4;
for( let i=0; i < channels; i++ ){
let i4 = i*4;
let r = image.data[i4 + 0];
let g = image.data[i4 + 1];
let b = image.data[i4 + 2];
greyImage.data[i4 + 0] = Math.round(0.21*r + 0.72*g + 0.07*b);
greyImage.data[i4 + 1] = g;
greyImage.data[i4 + 2] = b;
greyImage.data[i4 + 3] = 255;
}
return greyImage;
}
getRGBAFromImageByXY(imageData, x, y) {
let rowStart = y * imageData.width * 4;
let pixelIndex = rowStart + x * 4;
return [
imageData.data[pixelIndex],
imageData.data[pixelIndex+1],
imageData.data[pixelIndex+2],
imageData.data[pixelIndex+3],
]
}
Reload your browser (http://127.0.0.1:8080/), check 'Use Object Recognition' and click 'Start evolution' button. Again, you should see the games begin, but now get a lot of game.gameApi.getPlayerX is not a function errors. Ok, we are making progress, let's implement this method.
This method is actually the method we hook into to do all of the processing of the games canvas. Realistically we could have pulled this out into it's own setInterval(), but for the purpose of this demo let's couple it in with getPlayerX() which is called within every think() invocation.
Now add in a method to determine the players x/y position on the canvas (we do this by finding the 10x10 pixel that is color #000000 (black)). Simple, yet effective.
js/ai/GameImageRecognition.js
updatePlayerPositionFromVisualTrackingMap(visualTrackingMap, colors) {
for (const xy in visualTrackingMap) {
let value = visualTrackingMap[xy];
if ( colors.player == value) {
let position = xy.split('x');
this.#playerX = parseInt(position[0]);
this.#playerY = parseInt(position[1]);
// If we dont have a ground, then set it
if( 0 == this.#playerGroundY ) {
this.#playerGroundY = this.#playerY;
}
}
}
}
Next up is a lot of logic to look through visualTrackingMap which stores all the colors of each 10x10 pixel section and determine what lies ahead in relation to the player.
js/ai/GameImageRecognition.js
getSectionAhead(playerX, playerY, aheadIndex, pixelMapSize, playerGroundY){
let x;
let y;
let section;
let aheadWidth = aheadIndex*10;
x = Math.ceil(playerX/pixelMapSize) * pixelMapSize + aheadWidth;
y = Math.ceil(playerY/pixelMapSize) * pixelMapSize;
section = this.getCollisionSectionAhead(x, y);
if( false == section ) {
section = [x, playerGroundY+pixelMapSize, pixelMapSize, pixelMapSize];
}
return {
x: section[0],
y: section[1],
width: section[2],
height: section[3],
};
}
// Logic to get the xy and width/height of the section ahead that we need to use to determine if we jump over or not
getCollisionSectionAhead(x, y) {
// Look for drop/dip section ahead we need to jump over
y = this.#playerGroundY;
if ( this.isSectionSolid(x, y) ) {
// Look for taller section ahead we need to jump over
let xyStart = this.findTopLeftBoundsOfSolidSection(x, y-this.#visualTrackingMapSize);
let xyEnd = this.findTopRightBoundsOfSolidSection(xyStart[0], xyStart[1], 1);
return [xyStart[0], xyStart[1], xyEnd[0] - x, y - xyEnd[1] + this.#visualTrackingMapSize];
} else {
if ( false === this.isSectionSolid(x, y+this.#visualTrackingMapSize) ) {
let xyStart = this.findBottomLeftBoundsOfSolidSection(x, y);
let xyEnd = this.findBottomRightBoundsOfSolidSection(xyStart[0], xyStart[1], 1);
return [xyStart[0], xyEnd[1]+this.#visualTrackingMapSize, xyEnd[0] - x, this.#visualTrackingMapSize];
}
}
return false;
}
isSectionSolid(x, y){
let section = this.#visualTrackingMap[x + 'x' +y];
if ( this.#colors.block == section ) {
return true;
}
return false;
}
findTopLeftBoundsOfSolidSection(x, y) {
if ( this.isSectionSolid(x, y) ) {
return this.findTopLeftBoundsOfSolidSection(x, y-this.#visualTrackingMapSize)
}
return [x,y+this.#visualTrackingMapSize];
}
findTopRightBoundsOfSolidSection(x, y, counter) {
if ( counter < 5 && this.isSectionSolid(x, y) ) {
counter++
return this.findTopRightBoundsOfSolidSection(x+this.#visualTrackingMapSize, y, counter)
}
return [x,y];
}
findBottomLeftBoundsOfSolidSection(x, y) {
if ( false === this.isSectionSolid(x, y) && y < this.#visualTrackingCanvas.height) {
return this.findBottomLeftBoundsOfSolidSection(x, y+this.#visualTrackingMapSize)
}
return [x,y-this.#visualTrackingMapSize];
}
findBottomRightBoundsOfSolidSection(x, y, counter) {
if ( counter < 5 && false === this.isSectionSolid(x, y) ) {
counter++
return this.findBottomRightBoundsOfSolidSection(x+this.#visualTrackingMapSize, y, counter)
}
return [x,y];
}
getSectionFromPlayer(index) {
return {
x: this.#sectionAhead.x,
y: this.#sectionAhead.y,
width: this.#visualTrackingMapSize,
height: this.#playerY-this.#sectionAhead.y
};
}
I will be the first to admit the above logic is not clean/performant and can really be improved. But the purpose of this demo was to prove what is possible - feel free to submit a PR if you want to improve :-)
Getting closer... Reload your browser (http://127.0.0.1:8080/), check 'Use Object Recognition' and click 'Start evolution' button. You should now see a lot of game.gameApi.isPlayerJumping is not a function errors. Let's implement that and a few other methods that are needed regarding the player.
Reload your browser (http://127.0.0.1:8080/), check 'Use Object Recognition' and click 'Start evolution' button. You should now see a lot of game.gameApi.setDebugPoints is not a function errors. Let's implement.
We are actually missing a key method jump(). For the sake of this demo, we are just going to revert to calling the GameAPI. We could simulate this with a bit of trickery by focusing in on the canvas and triggering the 'spacebar' button. But a little too much for this demo.
js/ai/GameImageRecognition.js
jump(){
// The only way to simulate this is by pressing the spacebar key, but because we have multiple games at once it isn't easily possible.
this.#gameApi.jump();
}
Reload your browser (http://127.0.0.1:8080/), check 'Use Object Recognition' and click 'Start evolution' button. You should now see the game mostly work, although a few errors will pop up. Add in the below.
Reload your browser (http://127.0.0.1:8080/), check 'Use Object Recognition' and click 'Start evolution' button. Everything should work now, if you let it go it will eventually solve all of the levels.
However... Wouldn't it be nice to see what the ML is actually seeing on each game? Let's add in some debugging info
js/ai/GameImageRecognition.js
drawRectOnCanvas(rect, color) {
let context = this.#visualTrackingCanvas.getContext('2d');
context.beginPath();
context.strokeStyle = color;
context.lineWidth = "1";
context.rect(rect.x, rect.y, rect.width, rect.height);
context.stroke();
}
// Function responsible for drawing what the computer sees, we then use this to get the inputs for tensorflow
drawMachineVision() {
if( this.#enableVision ) {
// Clear everything first
this.#visualTrackingCanvas.getContext('2d').clearRect(0, 0, this.#visualTrackingCanvas.width, this.#visualTrackingCanvas.height);
// Draw player
this.drawRectOnCanvas({
x: this.#playerX,
y: this.#playerY,
width: this.#visualTrackingMapSize,
height: this.#visualTrackingMapSize,
}, this.#colors.visionOutline);
// Draw map sections
for (const xy in this.#visualTrackingMap) {
let value = this.#visualTrackingMap[xy];
if ( this.#colors.block == value) {
let position = xy.split('x');
this.drawRectOnCanvas({
x: parseInt(position[0]),
y: parseInt(position[1]),
width: this.#visualTrackingMapSize,
height: this.#visualTrackingMapSize
}, this.#colors.visionOutline)
}
}
this.drawRectOnCanvas({
x: this.#sectionAhead.x,
y: this.#sectionAhead.y,
width: this.#sectionAhead.width,
height: this.#sectionAhead.height,
}, 'blue');
}
}
Then change the method getPlayerX() to look like this.
Reload your browser (http://127.0.0.1:8080/), check 'Use Object Recognition', check 'Enable ML vision' and click 'Start evolution' button. You should now see lots of red boxes that highlight what the ML is actually using as inputs.
You're browser will most likely struggle, however it will work.
I get consistent results along the lines of:
Level 1: Takes 10-25 generations
Level 2: Takes 15-40 generations
Level 3: Takes 40-400 generations (as it has to learn to jump blocks and gaps).
Congratulations!
I hope this tutorial was useful for you to learn how to use TensorFlowJS to build a NeuroEvolution implementation. Any questions leave them in the comments below, or tweet me on twitter at @dionbeetson
We will be adding in useful debugging information to the right hand column and to the game itself.
Chart the progress of each evolution
We are going to use ChartJS to track the progress (as a percentage) and the score for the best game of each evolution. Create a file called js/ai/NeuroEvolutionChart.js and copy the below logic into it. It is a basic ChartJS implementation that I won't go into detail in this tutorial.
Now, add an update() method that will be responsible for pushing the data points onto the chart. It will also only keep 12 data points (the very first evolution, and the latest 11 evolutions). This is to avoid clutter on the page.
Reload your browser, and click the 'start evolution' button. You should start seeing debug information in your right hand column.
Another useful debug is to visualise the inputs that are being used to predict the jump() action.
Back in the Ai class within the think() method, just before the game.brain.predict() line, add in this code block to push debug points into the game canvas.
Reload your browser, and click the 'start evolution' button. You should start seeing crosshairs for the inputs being used by TensorFlow JS on every prediction call.
Sometimes it is useful to be able to 'pause the game before the next evolution begins. You will see a checkbox in the left column labelled 'Pause after generation'. Let's wire it up, as it currently is not.
In ui.js add in the below change handler to capture user interaction.
This isn't enough though, we need to build in the logic to actually pause the game. We are basically going to wrap the ai.start(bestPlayerBrainsByFitness); call within the start() method in NeuroEvolution.js in an IF statement based on if the pause checkbox is checked. If it is checked, we will then store the current games within a local class variable so we can retrieve it once the checkbox is unchecked.
The completed method looks like this now.
js/ai/NeuroEvolution.js
start(games, bestPlayerBrainsByFitness) {
this.updateUIRoundInformation();
if ( this.#generation < this.#maxGenerations ) {
if( false == this.#pauseBeforeNextGeneration ){
for ( let i = 0; i < games.length; i++ ) {
games[i].gameApi.remove();
}
games = undefined;
this.#pausedGames = [];
this.#pausedBestNeuralNetworksByFitness = [];
this.#generation++;
const ai = new Ai(this.finishGeneration.bind(this));
ai.start(this.#useImageRecognition, bestPlayerBrainsByFitness);
} else {
this.#pausedGames = games;
this.#pausedBestNeuralNetworksByFitness = bestPlayerBrainsByFitness;
for ( let i = 0; i < games.length; i++ ) {
games[i].gameApi.show();
}
}
} else {
this.enableSpeedInput();
}
}
Then add in the following variable declarations in the NeuroEvolution class.
And then add in the required getters/setters in the NeuroEvolution class.
js/ai/NeuroEvolution.js
get pauseBeforeNextGeneration() {
return this.#pauseBeforeNextGeneration;
}
set pauseBeforeNextGeneration( pauseBeforeNextGeneration ) {
this.#pauseBeforeNextGeneration = pauseBeforeNextGeneration;
}
get pausedGames() {
return this.#pausedGames;
}
get pausedBestNeuralNetworksByFitness() {
return this.#pausedBestNeuralNetworksByFitness;
}
get bestGames() {
return this.#bestGames;
}
set pauseBeforeNextGeneration( pauseBeforeNextGeneration ) {
this.#pauseBeforeNextGeneration = pauseBeforeNextGeneration;
}
Speeding up NeuroEvolution
Ok that is debugging done for now. Let's move onto improving our NeuroEvolution to solve this game a little faster. This involves 3 things:
Adding more inputs for TensorFlow to use to predict jumping
After each evolution randomly choosing to use the 'best game to date', as this can keep things on track if we have a single bad evolution randomly (trust me it happens)
Implementing a crossover method so instead of mutating only 1 of the best games, we take 2 of the best games and breed a child NeuralNetwork from their 2 results (which is called a crossover)
If a game passes the level, don't mutate or crossover, as we already have a successful NeuralNetwork
Adding more inputs for TensorFlow JS
In the Ai class we are going to add in 3 more inputs for (isPlayerJumping, playerVelocity & canPlayerJump). Within the think() method after the existing 5 inputs, add in these additional inputs.
Then change the variable declaration for inputs in Ai class to 8
js/ai/Ai.js
#inputs = 8;
Implement crossover within TensorFlowJS
In the NeuroEvolution class within the completeGeneration() method, we are going to modify the breeding section to make it a little smarter. Make the for loop look like this.
js/ai/NeuroEvolution.js
// Breeding
for (let i = 0; i < games.length; i++) {
let bestPlayerA = this.pickBestGameFromFitnessPool(games);
let bestPlayerB = this.pickBestGameFromFitnessPool(games);
let child;
child = this.mutateNeuralNetwork(this.crossoverNeuralNetwork(bestPlayerA.neuralNetwork.clone(), bestPlayerB.neuralNetwork.clone()));
bestPlayerBrainsByFitness.push(child);
}
Now we need to add in the crossover method to the same file.
js/ai/NeuroEvolution.js
crossoverNeuralNetwork(neuralNetworkOne, neuralNetworkTwo) {
let parentA_in_dna = neuralNetworkOne.input_weights.dataSync();
let parentA_out_dna = neuralNetworkOne.output_weights.dataSync();
let parentB_in_dna = neuralNetworkTwo.input_weights.dataSync();
let parentB_out_dna = neuralNetworkTwo.output_weights.dataSync();
let mid = Math.floor(Math.random() * parentA_in_dna.length);
let child_in_dna = [...parentA_in_dna.slice(0, mid), ...parentB_in_dna.slice(mid, parentB_in_dna.length)];
let child_out_dna = [...parentA_out_dna.slice(0, mid), ...parentB_out_dna.slice(mid, parentB_out_dna.length)];
let child = neuralNetworkOne.clone();
let input_shape = neuralNetworkOne.input_weights.shape;
let output_shape = neuralNetworkOne.output_weights.shape;
child.dispose();
child.input_weights = tf.tensor(child_in_dna, input_shape);
child.output_weights = tf.tensor(child_out_dna, output_shape);
return child;
}
Reload your browser, and click the 'start evolution' button. You should start seeing games solve themselves a bit faster in fewer generations (normally around 20).
10% chance to crossover best game
Back in the NeuroEvolution class, within breeding section, let's add one final update so that on random occasions (10% chance), we do a crossover with the best game to date. Make the breeding section look like this
js/ai/NeuroEvolution.js
// Breeding
for (let i = 0; i < games.length; i++) {
let bestPlayerA = this.pickBestGameFromFitnessPool(games);
let bestPlayerB = this.pickBestGameFromFitnessPool(games);
let bestPlayerC = this.#bestGames[0];
let child;
if ( random(1) < 0.1) {
const ai = new Ai();
let bestPlayerD = new NeuralNetwork(ai.inputs, ai.neurons, ai.outputs);
child = this.mutateNeuralNetwork(this.crossoverNeuralNetwork(bestPlayerC.neuralNetwork.clone(), bestPlayerD));
} else {
child = this.mutateNeuralNetwork(this.crossoverNeuralNetwork(bestPlayerA.neuralNetwork.clone(), bestPlayerB.neuralNetwork.clone()));
}
bestPlayerBrainsByFitness.push(child);
}
We also need to expose getters in the Ai class for the input, neuron & output variables. Add the following methods:
js/ai/Ai.js
get inputs(){
return this.#inputs;
}
get neurons(){
return this.#neurons;
}
get outputs(){
return this.#outputs;
}
Reload your browser, and click the 'start evolution' button. You should start seeing games solve themselves a bit faster in fewer generations (normally around 10-15).
Don't mutate on success
Now wrap the Breed functionality in an IF statement to ensure that if a game passes the level, we don't mutate or crossover, we just continue using the successful NeuralNetwork.
js/ai/NeuroEvolution.js
if ( false != gamePassedLevel ) {
for ( let i = 0; i < games.length; i++ ) {
if (games[i].gameApi.isLevelPassed() ) {
games[i].neuralNetwork.save('neuralNetwork');
for (let ii = 0; ii < games.length; ii++) {
bestPlayerBrainsByFitness.push(games[i].neuralNetwork.clone());
}
}
}
console.log('Level Passed:', this.#bestGames[0], this.#bestGames.length, this.#bestGames);
this.start(games, bestPlayerBrainsByFitness);
} else {
// Breeding
for (let i = 0; i < games.length; i++) {
let bestPlayerA = this.pickBestGameFromFitnessPool(games);
let bestPlayerB = this.pickBestGameFromFitnessPool(games);
let bestPlayerC = this.#bestGames[0];
let child;
if ( random(1) < 0.1) {
const ai = new Ai();
let bestPlayerD = new NeuralNetwork(ai.inputs, ai.neurons, ai.outputs);
child = this.mutateNeuralNetwork(this.crossoverNeuralNetwork(bestPlayerC.neuralNetwork.clone(), bestPlayerD));
} else {
child = this.mutateNeuralNetwork(this.crossoverNeuralNetwork(bestPlayerA.neuralNetwork.clone(), bestPlayerB.neuralNetwork.clone()));
}
bestPlayerBrainsByFitness.push(child);
}
this.start(games, bestPlayerBrainsByFitness);
}
Reload your browser, and click the 'start evolution' button. Once enough evolutions process to pass the level, you will notice that all future games in evolutions will start passing consistently.
This is a far as we will take this tutorial in regards to evolving and speeding up our NeuroEvolution logic. Next we will move onto saving/loading our NeuralNetwork models.
Saving & Loading our NeuralNetwork models
The last piece to implement is the saving and loading functionality of our NeuralNetwork. This is so once you have a NeuralNetwork that works you can save it and load it back when needed. We will be implementing the:
Ability to automatically save a NeuralNetwork that passes to localStorage
Ability to manually save best NeuralNetwork to localStorage
Ability to manually save best NeuralNetwork to disk
Ability to load from local storage
Ability to load from disk
First we need to add the ability to save within the NeuralNetwork.js file. Add these 2 methods
Automatically save a NeuralNetwork that passes to localStorage
In the NeuroEvolution. class within the method finishGeneration(). Add in the below to the IF statement when a game level is passed.
js/ai/NeuroEvolution.js
games[i].neuralNetwork.save('neuralNetwork');
So it should look like this:
js/ai/NeuroEvolution.js
for ( let i = 0; i < games.length; i++ ) {
if (games[i].gameApi.isLevelPassed() ) {
games[i].neuralNetwork.save('neuralNetwork');
for (let ii = 0; ii < games.length; ii++) {
bestPlayerBrainsByFitness.push(games[i].neuralNetwork.clone());
}
}
}
console.log('Level Passed:', this.#bestGames[0], this.#bestGames.length, this.#bestGames);
this.start(games, bestPlayerBrainsByFitness);
Your game will now automatically save the NeuralNetwork model to localStorage key neuralNetwork if it passes the level.
Manually save best NeuralNetwork to localStorage
To manually save to local storage add the below code to ui.js
To manually save to disk you can add the below code to ui.js
js/ai/ui.js
document.querySelector("#btn-ml-save-neuralnetwork-disk").addEventListener('click', function (event) {
if( neuroEvolution.bestGames.length > 0 ) {
let neuralNetworkJSON = neuroEvolution.bestGames[0].neuralNetwork.stringify();
let a = document.createElement("a");
let file = new Blob([neuralNetworkJSON], {type: 'application/json'});
a.href = URL.createObjectURL(file);
a.download = 'neuralNetwork.json';
a.click();
}
});
Loading NeuralNetwork Models
To load from local storage or disk we need to create a method to handle loading a NeuralNetwork into memory. Create the below method in ui.js (there is probably a better file location for this method now that I think about it).
js/ai/ui.js
const loadNeuralNetwork = (neuralNetworkData) => {
const ai = new Ai();
let neuralNetwork = new NeuralNetwork(ai.inputs, ai.neurons, ai.outputs);
neuralNetwork.dispose();
neuralNetwork.input_weights = tf.tensor(neuralNetworkData.input_weights);
neuralNetwork.output_weights = tf.tensor(neuralNetworkData.output_weights);
let neuralNetworks = [];
for ( let i = 0; i < ai.totalGames; i++ ) {
neuralNetworks.push(neuralNetwork);
}
let games = neuroEvolution.pausedGames;
neuroEvolution.pausedGames;
neuroEvolution.reset();
neuroEvolution.start(games, neuralNetworks);
};
We also need to add the below method into the Ai class.
js/ai/Ai.js
get totalGames(){
return this.#totalGames;
}
Load from local storage
Now to load from localStorage into memory, add the below into ui.js
js/ai/ui.js
document.querySelector("#btn-ml-load-neuralnetwork-localstorage").addEventListener('click', function (event) {
cachedLevelSections = [];
let games = neuroEvolution.pausedGames;
neuroEvolution.pausedGames;
neuroEvolution.reset();
let neuralNetworkData = localStorage.getItem('neuralNetwork');
loadNeuralNetwork(JSON.parse(neuralNetworkData));
});
We also need to add the reset method into the NeuroEvolution class.
Now to load from disk into memory, add the below into ui.js
js/ai/ui.js
document.querySelector("#btn-ml-load-neuralnetwork-disk").addEventListener('change', function (event) {
let file = this.files[0];
let reader = new FileReader()
let textFile = /application\/json/;
let fileText = '';
if (file.type.match(textFile)) {
reader.onload = function (event) {
let importedNeuralNetwork = JSON.parse(event.target.result);
loadNeuralNetwork(importedNeuralNetwork);
}
}
reader.readAsText(file);
});
Reload your browser, and click the 'start evolution' button. You will notice if you inspect Chrome DevTools you will see a localStorage key called 'brain' once a level is passed.
You can also now also use the buttons in the left hand menu to save and load the best NeuralNetwork to disk.
Within the GitHub codebase within the models/ directory you will see 3 existing models that can be loaded up to pass levels 1, 2 & 3.
Try it yourself and get your NeuroEvolution implementation to pass Level 1, then Level 2, then finally Level 3.
I get consistent results along the lines of:
Level 1: Takes 5-20 generations
Level 2: Takes 5-20 generations
Level 3: Takes 40-700 generations (as it has to learn to jump blocks and gaps).
Congratulations!
I hope this tutorial was useful for you to learn how to use TensorFlowJS to build a NeuroEvolution implementation. Any questions leave them in the comments below, or tweet me on twitter at @dionbeetson
This tutorial will walk you through how to implement NeuroEvolution using TensorFlow JS. This NeuralNetwork will learn how to play a simple browser based game that requires a player to jump over blocks and gaps. It is based on the source code found on my GitHub account here https://github.com/dionbeetson/neuroevolution-experiment.
The difference in this solution compared to other solutions out there is that instead of building ML directly into the core game source code (like I saw in a lot of examples); the ML and Game source code are completely decoupled. This simulates a much more realistic real world example, as you would rarely have direct access to the source code when applying ML (you would only have access to specific inputs). I created an an API/SDK for the browser based game to expose some key functionality (the inputs the NeuralNetwork needs), but end state is to use image analysis so this solution is more adaptable to other applications.
References
Although I wrote this codebase myself I took a lot of input/influence from a huge range of great tutorials that already exist. You can see a list of the key tutorials/repos I learned from on the GitHub Readme for this project here.
Then run in terminal the below to setup the repository for this tutorial:
npm install
npm run setup-tutorial
This basically moves the js/ai directory to js/ai-source, and leaves everything else in place. If you ever need to refer back to the original source code use js/ai-source as reference.
Now run the below to startup the app:
npm run dev
The browser will load up the application, but you will see 404 errors within the DevTools console. You will though however, have a working browser game. Click 'Play game', and use the spacebar to jump to test things out. Throughout this tutorial you are going to be building a NeuroEvolution implementation to learn how to automatically solve this game.
Quick overview of the UI
Left section: Includes base controls for Manual and AI interactions. In order from top to bottom
Level: You can select the level (there are currently 3 levels available in the game in order of complexity)
Play game: Play an user interacted game
Enable drawing: Whether you want to render the drawing on canvas
Speed: How fast the game will run
Start Evolution: Start the NeuroEvolution
Save to localStorage: Save best current NeuralNetwork into localStorage
Save to disk: Save best current NeuralNetwork to disk
Load from localStorage: Load current NeuralNetwork in local storage into memory and start NeuroEvolution
Load from disk: Load a NeuralNetwork from disk into memory and start NeuroEvolution
Pause after generation: For debugging, pause after each evolution so you can see / debug it
Middle section: The game(s) render within the large middle column
Right section: Is used to display debugging information for the NeuroEvolution implementation
This tutorial has 4 key files
For the entire tutorial we will be working within the js/ai/ directory.
ui.js: Controls the UI interactions
NeuralNetwork.js: Is the base neural network
Ai.js: This class instantiates (by default 9) games & corresponding NeuralNetwork instances
Neuroevolution.js: This class will run up to 1500 generations of Ai.js until it successfully passes the selected level
Setup base structure
Let's start to build out enough base logic to be able to predict if we should jump or not using TensorFlowJS.
Create a directory js/ai, and create a file called js/ai/ui.js which will handle basic UI controls. Within this file we want to instantiate the NeuroEvolution class (that we are about to create, and bind a UI button to start the evolution process).
js/ai/ui.js
let neuroEvolution = new NeuroEvolution();
document.querySelector("#btn-ml-start").addEventListener('click', function (event) {
neuroEvolution.start([]);
}, false);
Now we need to create the skeleton of NeuralNetwork.js, Ai.js & Neuroevolution.js.
For the NeuralNetwork let's just start with a constructor that allows us to send input_node/hidden_node & output_node count arguments.
js/ai/NeuralNetwork.js
class NeuralNetwork {
constructor(input_nodes, hidden_nodes, output_nodes) {
// The amount of inputs (eg: player y position, height of next block etc..)
this.input_nodes = input_nodes;
// Amount of hidden nodes within the Neural Network)
this.hidden_nodes = hidden_nodes;
// The amount of outputs, we will use 2 (will be needed for level 3)
this.output_nodes = output_nodes;
// Initialize random weights
this.input_weights = tf.randomNormal([this.input_nodes, this.hidden_nodes]);
this.output_weights = tf.randomNormal([this.hidden_nodes, this.output_nodes]);
}
}
Now we need to create the Ai class. This will include a method that will create 1 game for now and link it with a NeuralNetwork instance. We will start with a start() method to create instances of a NeuralNetwork, a checkGame() method that will be called every 50ms which will call the think() method that we will expand to make the actual prediction to jump or not jump.
js/ai/Ai.js
class Ai {
#totalGames = 1; // Will increase to 9 games later
#inputs = 5;
#neurons = 40;
#outputs = 2;
#games = [];
#gamesRunning = 0;
#sectionsToSeeAhead = 1;
#timeTakenDateStart = null;
start(useImageRecognition, neuralNetworks, completeCallback) {
this.#timeTakenDateStart = new Date();
for ( let i = 0; i < this.#totalGames; i++ ) {
let neuralNetwork;
if ( undefined !== neuralNetworks && neuralNetworks[i] instanceof NeuralNetwork ) {
neuralNetwork = neuralNetworks[i];
} else {
neuralNetwork = new NeuralNetwork(this.#inputs, this.#neurons, this.#outputs);
}
let gameApi;
if ( useImageRecognition ) {
gameApi = new GameImageRecognition();
} else {
gameApi = new GameApi();
}
this.#games[i] = {
gameApi: gameApi,
neuralNetwork: neuralNetwork,
interval: null
}
// Debug look ahead
this.#games[i].gameApi.setHighlightSectionAhead(this.#sectionsToSeeAhead)
// Start game
this.#gamesRunning++;
this.#games[i].gameApi.start();
this.#games[i].interval = setInterval(this.checkGame.bind(null, this, this.#games, this.#games[i]), 50);
}
}
checkGame(ai, games, game) {
if( game.gameApi.isSetup() ) {
ai.think(game);
}
}
think(game) {
}
}
Now create the NeuroEvolution class that will be responsible for running multiple evolutions of Ai.js until we get an evolution that passes the level.
js/ai/NeuroEvolution.js
class NeuroEvolution {
#generation = 1;
#maxGenerations = 1500;
#useImageRecognition = false;
start(games, bestPlayerBrainsByFitness) {
if ( this.#generation < this.#maxGenerations ) {
for ( let i = 0; i < games.length; i++ ) {
games[i].gameApi.remove();
}
games = undefined;
this.#generation++;
const ai = new Ai(this.finishGeneration.bind(this));
ai.start(this.#useImageRecognition, bestPlayerBrainsByFitness);
} else {
this.enableSpeedInput();
}
}
finishGeneration(games, timeTaken) {
// TODO
}
updateUIRoundInformation() {
document.querySelector("#round-current").innerHTML = this.#generation;
document.querySelector("#round-total").innerHTML = this.#maxGenerations;
document.querySelector("#round-progress").style.width = '0%';
document.querySelector("#generation-progress").style.width = (this.#generation/this.#maxGenerations)*100 + '%';
}
}
Now you have your base structure in place.
Reload your browser (http://127.0.0.1:8080/), and click the 'Start evolution' button. You should see 1 game playing, and failing (no jumping), no matter how many times you reload the page. That is because we haven't implemented the prediction logic. Let's implement that using TensorFlowJS in a moment.
Quickly, before that, in the Ai class, let's create 9 games at once instead of 1, as NeuroEvolution will be incredibly slow with only 1 NeuralNetwork at a time. Depending on your computer specs you can increase this to a higher number. I did find that 9 worked quite well though.
js/ai/Ai.js
#totalGames = 9
Reload your browser, and click 'Start evolution' button again. You should see 9 games playing.
Predicting whether to jump or not
To use TensorflowJS to predict to jump, we need to modify a few files. First in Ai.js in the think() method add the below logic in. Essentially we are accessing public information that the JS game exposes in js/game/GameAPI.js (like player position, next block height etc..) and then normalising all inputs to be between 0-1 as TensorFlow JS requires this.
js/ai/Ai.js
think(game) {
let inputs = [];
let inputsNormalised = [];
// Player y
inputs[0] = (game.gameApi.getPlayerY());
inputsNormalised[0] = map(inputs[0], 0, game.gameApi.getHeight(), 0, 1);
// Player x
inputs[1] = game.gameApi.getPlayerX();
inputsNormalised[1] = map(inputs[1], inputs[1], game.gameApi.getWidth(), 0, 1);
let section = game.gameApi.getSectionFromPlayer(this.#sectionsToSeeAhead);
// 2nd closest section x
inputs[2] = section.x + section.width;
inputsNormalised[2] = map(inputs[2], inputs[1], game.gameApi.getWidth(), 0, 1);
// 2nd closest section y
inputs[3] = section.y;
inputsNormalised[3] = map(inputs[3], 0, game.gameApi.getHeight(), 0, 1);
// 2nd closest section y base
inputs[4] = section.y + section.height;
inputsNormalised[4] = map(inputs[4], 0, game.gameApi.getHeight(), 0, 1);
// Call the predict function in the NeuralNetwork
let outputs = game.neuralNetwork.predict(inputsNormalised);
// If input is about 0.5 then jump, if not, stay still
if ( outputs[0] > 0.5 || outputs[1] > 0.5 ) {
game.gameApi.jump();
}
}
We also will need to create the predict() method in the NeuralNetwork class. This method creates an input_layer, hidden_layer and output_layer and returns 2 predictions (which are both numbers between 0-1).
js/ai/NeuralNetwork.js
predict(user_input) {
let output;
tf.tidy(() => {
let input_layer = tf.tensor(user_input, [1, this.input_nodes]);
let hidden_layer = input_layer.matMul(this.input_weights).sigmoid();
let output_layer = hidden_layer.matMul(this.output_weights).sigmoid();
output = output_layer.dataSync();
});
return output;
}
Reload your browser, and click the 'start evolution' button.
You will now see 9 games playing, and some should be jumping, and some not. This is completely random. Reload a few times if you need to to ensure this is the behaviour you are seeing.
Once all of your players die you will notice the game stops. As of now, we have used TensorFlow JS to make a prediction about whether to jump or not. This is based on a set of inputs (player X, player Y & section ahead height/Y).
Now we need to add in the NeuroEvolution aspect so that we pick the best progressed games and start a new evolution with those NeuralNetworks (but we will evolve/mutate them slightly before hand).
In NeuroEvolution add a method called calculateFitness(). This method iterates through all games once finished, creates a fitness rating which is just the % progress of the game. It then has logic to increase the fitness of better performing games, so that later on in our logic, there is a higher chance the better performing NeuralNetworks are chosen for the next evolution. Hope that all makes sense.
js/ai/NeuroEvolution.js
calculateFitness(games) {
for ( let i = 0; i < games.length; i++ ) {
let game = games[i];
games[i].fitness = game.gameApi.getProgress() / 100;
games[i].score = game.gameApi.getScore();
games[i].progress = game.gameApi.getProgress();
}
// The below code makes the better progressed games have a higher fitness so they have a higher chance of being selected for next generation
games.sort(this.sortByFitness);
games.reverse();
let prev = 0;
for ( let i = 0; i < games.length; i++ ) {
games[i].fitness = this.#discountRate * prev + games[i].fitness;
prev = games[i].fitness;
}
games.sort(this.sortByFitness);
return games;
}
In NeuroEvolution.js add a method that will check if at least 1 game has passed the level in the current evolution.
js/ai/NeuroEvolution.js
didAtLeastOneGameCompleteLevel(games) {
for ( let i = 0; i < games.length; i++ ) {
if (games[i].gameApi.isLevelPassed() ) {
return games[i];
}
}
return false;
}
In NeuroEvolution add a method that will pick the exact best player with the highest fitness. We use this to display and keep a track of the best performing NeuralNetwork.
js/ai/NeuroEvolution.js
pickBestGameByActualFitness(games){
let game;
let prevFitness = 0;
for ( let i = 0; i < games.length; i++ ) {
if (games[i].fitness > prevFitness) {
game = games[i];
prevFitness = game.fitness;
}
}
return game;
}
In NeuroEvolution add a method that will pick one (not always the best like the above method) of the best players with the highest fitness
js/ai/NeuroEvolution.js
pickBestGameFromFitnessPool(games) {
let index = 0;
let r = random(1);
while (r > 0 ) {
if( undefined !== games[index] ) {
r = r - games[index].fitness;
index++;
} else {
r = 0;
}
}
index--;
let game = games[index];
return game;
}
In NeuroEvolution we can now improve the finishGeneration() method with the below logic. This code will:
Calculate fitness for all 9 finished games
Store the best player in a local variable
Add the best player to the bestGames[] array
Ensure we only keep the best 5 games (for memory/performance reasons)
Iterate over all games and mutate the best performing NeuralNetworks into new NeuralNetworks for the next evolution
Start the next NeuroEvolution
js/ai/NeuroEvolution.js
finishGeneration(games, timeTaken) {
games = this.calculateFitness(games);
// Did one of the games finish?
let gamePassedLevel = this.didAtLeastOneGameCompleteLevel(games);
let bestPlayerByFitness = gamePassedLevel;
let bestPlayerBrainsByFitness = [];
if( false === bestPlayerByFitness ){
bestPlayerByFitness = this.pickBestGameByActualFitness(games);
}
this.#bestGames.push(bestPlayerByFitness);
this.#bestGames.sort(this.sortByFitness);
// Only keep top 5 best scores
if( this.#bestGames.length > 5 ) {
this.#bestGames = this.#bestGames.slice(0, 5);
}
// Breeding
for (let i = 0; i < games.length; i++) {
let bestPlayerA = this.pickBestGameFromFitnessPool(games);
let child;
child = this.mutateNeuralNetwork(bestPlayerA.neuralNetwork.clone());
bestPlayerBrainsByFitness.push(child);
}
this.start(games, bestPlayerBrainsByFitness);
}
Lastly, in NeuroEvolution you will need to add the below variable declarations.
js/ai/NeuroEvolution.js
#discountRate = 0.95;
#bestGames = [];
Reload your browser, and click the 'start evolution' button. You should again see 9 games playing, some will also be jumping like last time. However, it still wont progress past the first evolution. This is because we need to wire up Ai.js to call the callback method in the checkGame method and add some logic to check if the game is over. Let's update the checkGame() method to look like the below.
Reload your browser, and click the 'start evolution' button.
Ok, we are making progress, but we are now missing the clone() method. We need to be able to clone NeuralNetworks after each generation so we don't interfere with previous NeuralNetwork objects.
In NeuralNetwork class add the clone() method as well as the dispose() method, as that is called from within the clone() method.
Reload your browser, and click the 'start evolution' button. You will see we are close now, we are just missing the mutateNeuralNetwork() method. This method is responsible for slightly tweaking the cloned NeuralNetwork so that you get a slightly different result in the next evolution. Let's go ahead and add the mutateNeuralNetwork() method into the NeuroEvolution class. I'm not going to go into more detail on this, but you can google it and find this method in a few other examples out there. I also wasn't the original author of this, I just adjusted it to work for my implementation. See GitHub for reference to original source code.
js/ai/NeuralEvolution.js
mutateNeuralNetwork(b) {
function fn(x) {
if (random(1) < 0.05) {
let offset = randomGaussian() * 0.5;
let newx = x + offset;
return newx;
}
return x;
}
let neuralNetwork = b.clone();
let ih = neuralNetwork.input_weights.dataSync().map(fn);
let ih_shape = neuralNetwork.input_weights.shape;
neuralNetwork.input_weights.dispose();
neuralNetwork.input_weights = tf.tensor(ih, ih_shape);
let ho = neuralNetwork.output_weights.dataSync().map(fn);
let ho_shape = neuralNetwork.output_weights.shape;
neuralNetwork.output_weights.dispose();
neuralNetwork.output_weights = tf.tensor(ho, ho_shape);
return neuralNetwork;
}
Reload your browser, and click the 'start evolution' button. Now after each of the 9 games fail the first time, the UI will start a new evolution. If you let this go for a few minutes you should see the game get solved - it normally takes around 30-50 generations with this current implementation.
Congratulations!
This is a very simple implementation of NeuroEvolution. In Part 2, we will go through making improvements to really speed things up including:
Implementing crossover within TensorFlowJS
Using the best progressing NeuralNetwork to ensure faster NeuroEvolution
At the start of 2016 myself and Juan (a friend and colleague) decided it would be a great idea to present on NewsCorp Australia's migration to WordPress at WordCamp Europe in Vienna. We had been through a huge migration project over the previous 18 month, moving Australia's largest media websites over to an enterprise version of WordPress.
Jump forward 2 months and we found out we were accepted, followed by 2 months of planning and cramming 18 months of project work into a 30 minute presentation. Before you know it we were flying out to Vienna from Sydney Australia for a quick 3 day trip at WordCamp Europe - the largest WordCamp event in the world.
Below is the talk we presented and transcript on how we scaled WordPress to support over 3 million posts across 22 sites, with over 16 million users a day.
Transcript
00:00
18 months ago, News Corp Australia, started its journey migrating all of its websites over to WordPres. Both Juan here and myself were on the software engineering team part of that migration project. We had often heard at the start of this project, WordPress doesn't scale. Well our talk today is to prove them wrong, and to demonstrate that WordPress provides a really good base. It's a really solid platform, and if you implement the correct technical solutions to extend WordPress, it can scale really well for large and high volume traffic websites like ours.
00:48
Historically people may have interpreted News Corp Australia to be a little like this, a 90 year old company, potentially focusing a lot on their newspaper side of the business. But over the past 18 months, a few of the people you can see in this photo plus a lot more in News Corp, migrated 22 of Australia largest sites including news.com.au, theaustralian.com.au & foxsports.com.au over to WordPress VIP. We've imported over 3 millions posts, and are receiving around 16 million users a day.
01:26
Why did we move to WordPress VIP? First of all, what is WordPress VIP? WordPress VIP is a hosting option for WordPress sites that is powered by wordpress.com which is an enterprise level hosting. We needed to escape our old CMS, which was unscalable and very expensive to maintain. WordPress was chosen because it is a very reliable tool, and most of our journalists are already familiar with, making training and up skilling across thousands of journalists much easier. We just needed to find a way to scale WordPress to meet the needs of our business.
02:06
Lastly and possibly the best part, with WordPress VIP we didn't need to worry about this. wordpress.com manages our production infrastructure, so updates to the core get directly deployed, we don't need to worry about updating core, or server patches. Our main focus is developing the best new features for our customers.
At a high level we have developed 45 WordPress plugins and 22 themes. apart from our production infrastructure, we run 58 AWS non production environments, which are a combination of EC2, RDS and SQS, where we try to mimic as much as technically possible the Wordpress.com environments. From a deployment perspective, we run 25 deployments to production each week through our CI/CD platform; and we love our unit tests, we have over 5000 and counting.
03:21
Our journey started in November 2014, when we look back at those 18 months, there are 5 key factors that really challenged us. We wish we had of known about these before we started the project. however, encountering these challenges and discovering their solutions were part of our journey, and actually a great team building exercise. I'm not going to lie though, it was a tough tough journey.
We wanted to share our story for 2 main reasons. First, to demonstrate that WordPress can scale for the enterprise, and secondly, to share the challenges we faced and the solutions we implemented.
4:03
We are going to focus on 5 key areas; site build; front-end; content ingestion: authoring - how we are creating and updating new stories; as well as continuous integration and deployments. We are going to talk about the challenges we faced and the solutions we implemented.
4:24
The first talking point, side build. We really needed the most simplistic way for our WordPress admins to manage and organise the content on our sites. Customizer was a great start to this, it provided a really good base, but we needed to extend it, and this is the functionality we are going to take you through. Working really closely with the XWP team who I can see a lot of in the front here. we added a lot of functionality into customizer, a fair bit of which has been moved into WordPress core. Here is a short video demonstrating just some of the few key features that have been added in or may already now exist in WordPress core.
05:02
All of our sites utilize multiple content areas - that is pretty standard stuff. Within each of our content areas, we have this concept of default widgets, or also known as global widgets, which will render on every single page. We are currently on the business page, and we are looking at the right hand rail, it's just a list of stories. If we now go to the news section in the navigation, you will notice that the widgets we just saw and actually the same widgets that were just rendered. So we can spin up a lot of pages that have very similar structure very quickly. Then to extend that, there is a plugin which we developed which allows you to override the widgets on a specific page and content area. We are on the homepage now, and by checking the 'localise to the current page' checkbox, it allows your to add a completely different set of new widgets for a specific content area for that specific page.
05:54
We call these contextual settings and they are stored as a JSON data structure as a custom post type in WordPress. The next thing you are going to see is our homepage, and that is the next big challenge we had. We have a lot of widgets that keep scrolling, and scrolling, and it's maybe a bit too much content, and that's what caused us a few challenges. By default WordPress as I'm sure a few of us know, stores all of its widget data in sidebars which is then stored in WordPress options. Because we have a huge huge homepage, it meant that the amount of data we were storing in WordPress options was more than 1mb. We leverage Memcache quite extensively, and 1mb is unfortunately the limit in Memcache, so as soon as any key has more than 1mb of data, it's not longer getting cached. This meant that for every single page request, we weren't caching WordPress options, which was a huge performance issue for us.
Another plugin was developed, we called this widgets-plus. Essentially it's a new custom post type and it stores all of the widget data now in its own custom post type in WordPress. This means that we can store everything relevant to just a single widget in isolation, and cache it in isolation, we don't need to worry about caching everything as part of WordPress options. Within our previous CMS it took us hours, sometimes a day to build a relatively simple page. within WordPress, we only showed you a very small part of it, but we can build really complex pages in literally a few minutes. It's a huge efficiency boost for our business.
07:24
After a few months we had a solution for site build, now the front-end architecture was really key for us. We needed to make sure when all of the sites were moving across into WordPress, we weren't going to end up with a mess of spaghetti front-end code.
07:38
I'm sure all of you here like us who are developers (front-end or backend) know that PHP as a templating language, isn't necessarily the best approach. It gets really messy really quickly. We needed a templaitng engine to ensure we decoupled our front-end code from our backend. Twig was our templating engine of choice, and there are 2 plugins that power this implementation. The first one is called vip-twig, and this one is responsible for integrating Twig into WordPress core - it is a very standalone plugin. Our next plugin, template-integration is responsible for exposing all of those core WordPress functions into our twig templates. It basically bridges the gap between WordPress core and the data our templates need to render out posts on the front-end. The good part about this is if we need to implement a new templating language because twig is no longer the laungauge to use, we can keep our template-integration plugin as it's all reusable, and we can simply build a new vip-smarty or vip-ejs plugin to manage that integration for that new templating engine.
08:40
We have now built our sites, and we have a templating engine in place, now we need content. We have been working on content ingestion functionality for around 6 months in parallel, and in May 2015, we were moving into beta testing in a VIP production environment, planning on launching our first site in June 2015. As you can see from our slides, the timeline is no where near the far right of the screen - yes we ran into some problems.
09:14
In our first attempt we ran into 2 major problems. First our posts are authored in a different platform, they are not authored in WordPress, and we had a benchmark of 60 seconds that content needed to be published into the WordPress front-end from the moment it was saved. Secondly we were seeing some rare race conditions in content ingestion, which caused content to be duplicated on our WordPress sites.
As well as the above 2 major problems, when we moved into production we needed to do an initial content import of around 300,000 posts per site. We didn't have MySQL access as our sites are hosted on wordpress.com - we couldn't expect them to give us access. It was a challenge to import that amount of information. Lastly, after that we, would have a constant stream of data updates including news stories, videos, photos coming into our sites at a rate of around 16 updates per second.
10:28
We needed an architecture to support this. Here is a simplified diagram (it is a very simplified diagram) of what we have implemented. It can be read from bottom to top. First our journalist will create a story or update it in one of our editorial tools. This post will then be sent to our API platform which manages all of our content. It will organise it, categorise it, and make it available to other platforms to consume it, WordPress being one of those platforms. On top of that, as soon as a message arrives into the API platform, it will send a notification to our SQS system in WordPress informing it of the new story and to 'start' ingesting.
11:18
With the help of the WordPress VIP team, we developed an ingestor daemon, we call it Turbo, because in the beginning it was built using WordPress crons, but wp-crons were not meant to work in that way. Like I said before, we were doing 16 updates a second and wp-cron were not coping with that, they were not fast enough. We needed to create this multi-threaded daemon. This whole process that you are seeing is what we call 'end to end publishing'. With this architecture in place, we have a single entry point for content imports, whether we are doing 300,000 updates in an initial import, or 16 updates a second. We also have with SQS, reduced the amount of duplicate content, actually it is now at zero, and finally we can ingest content in less than 60 seconds, we are basically ingesting content in less than 10 seconds, as soon as a journalist saves a story, it appears on our site within about 6 seconds.
12:38
Another challenge that we faced was the amount of fields and data we had that represented a 'NewsStory', or a 'WordPress post in our business. This is stored in our APIs, this is the object we get. As you can see there is a trimmed down version of how our JSON object looks, it's massive, it's a really big object. At the beginning we thought lets split this into pieces, save everything into post meta fields, and build them up again to send writes back to our APIs. However, we thought, why re-invent the wheel, if we are already using this JSON object across all our other platforms, let's just save it in our WordPress database, and consume it from there, and read it from there. Due to this we decided to store this JSON object in the post_content field in the posts table. With this architecture, we needed to change how our templates work, and how our plugins work with this field. We are no longer reading it as text field, but as a JSON object.
Saving the JSON object here allowed us to have revisions of an entire post object. Also when rendering a post on the page, we can do it with a single database call. We finally kept a simple rule, we would only use post_meta for searchable parameters.
All this gave us a huge performance benefits by more than halving the amount of database calls.
14:19
As you can probably guess, June came and past and we didn't launch our sites. But they stayed in beta testing for a few months. Our internal development team, the XWP team, and WordPress VIP team, worked for around 2 months to implement solutions to all the problems we just discussed; and one night in September, a very very late night, we launched our first site on WordPress VIP. It was a night filled with anxiety, way too much pizza, and then a sigh of relief when it switched on and everything just worked.
By now we have implemented a simplistic site build approach, our front-end is clean, and we have content quickly ingesting into our WordPress sites.
15:21
Now we needed to use WordPress for what it is meant to be used for; for editing and creating content. We developed these authoring screens within WordPress that expose our data JSON structure that defines our News story content so people can edit it in the most simplistic way. As you can see we leveraged meta boxes quite extensively, and used CSS and Javascript to provide the look and feel we were after. We tried not to deviate from WordPress core at all so any updates that come through to WordPress.com did not affect us.
15:58
Here is another image of one of our authoring screens to demonstrate what we are doing for image management within posts. We are leveraging off the media element in WordPress and allowing authors to edit/crop images within the UI.
16:15
Our category management is also a little different, our stories have to have in the permalink, the URL of the specific category they belong to. So we needed to add in a primary category (or section as we call it), that they have to select when creating a story. Then you can put that story in multiple categories if needed.
16:48
Within the first week of February in 2016, we launched our last 4 sites into wordpress.com in just one week. And everything just rolled out very very smoothly. Then gradually over the next 4 months, until pretty much last week, all that authoring functionality that Juan just showed you, rolled out into a production environment.
17:08
So we needed to maintain this complex WordPress platform that we had setup. We also needed to ensure that at all costs it was impossible for any cowboy coders to rise up and try and take control of our codebase.
For this it was really critical that we had a solid continuous integration and deployment platform setup. We needed to make sure that a developer can get his code or her code from their local environment all the way into production in the most stable and risk free way.
17:31
For us it comes down to feature based branching and deployments. Every feature for sits within it's own feature branch, branched off master in GIT. It's always developed in isolation, it's always tested in isolation, it's always deployed in isolation. Our team chose to never bundle up more than one feature and deploy it at once, as for us this added too many risks and too many dependencies.
Our team then utilizes pull requests to control any code merges into master. This is where code reviews happen, which is one of our favourite pastimes. Every single line of code is code reviewed by another developer. No single line of code reaches production without being code reviewed by at least 1, 2 or sometimes 3 other developers in the team.
18:14
We then utilize Atlassians Bamboo continuous integration platform to manage all of our automation. This is where on every code commit we run PHPCS to make sure all of our coding standards are being met, we run PHPUnit to make sure all of our unit tests are passing and there are no regression issues we are aware of. Once all of those tests pass we package up the application, running any front-end build tasks in gulp or grunt, we compile our TWIG templates, then we deploy that out to our 50+ non-production environments.
This is when testing happens, either manual regression testing, or we have a suite of selenium based automation tests using the robot framework. Once all manual and automated tests have passed we deploy this feature through our continuous integration platform all the way into the WordPress VIP environment. We celebrate with a few beers, and then we start that process all over again for the next feature.
The best part is, this isn't the process we started with 18 months ago. This evolved over countless retrospectives, team huddles, and big issues with our previous processes, to what it is today.
19:48
With a solid continuous integration and deployment platform setup, the only other challenge we faced was with WordPress VIP you cannot control the exact time your code is deployed to production, and if your feature has changes in 3, 4 or 5 different code repositories, you can't guarantee the order the code in those repositories are always deployed in.
19:32
We needed a way to be able to turn features on and off in a production environment. Welcome to the feature toggle, This was developed by one of our developers called Roman back in Australia, and it is an implementation of the feature toggle technique by Martin Fowler. It abstracts all of the complexities around feature toggling. So with just a few lines of code you can setup a new feature, you can give it a nice descriptive name, and in the last parameter (the boolean) you can decide if it is active or deactivated by default.
20:00
Then all of our themes and plugins have access to check if this feature is enabled or not. If it is enabled you can execute a certain chunk of code. Lastly this plugin exposes an admin page in WordPress, that lists all of the features we currently have available to us, it's de-active and activated state, and this is where you can toggle a certain feature on or off. This feature toggle has allowed us to, for our really big features, to roll them out to all of our sites from a deployment perspective, and then gradually toggle them on one site at a time so we can stagger the regression testing required.
20:32
So we have come to the end, and what have we really learnt? First of all, as Juan pointed out, migration projects are hard hard work. No matter matter how much planning you do, you are always going to run into unknowns. For us it was really critical for us to get our code into a beta version on the WordPress VIP infrastructure, so we could test our end to end solutions in their environment, rather than our non-production setup.
To all those people that said to us that WordPress isn't going to scale for what we are doing. Well if you just download WordPress core, install it on a single server an try and support 16 million uses, then well no it isn't going to work. But if you implement the correct solution architecture, with things like memcache, elastic search, asynchronous content ingestion, it scales really well for sites like ours.
21:21
Lastly, us partnering with two really awesome teams. With XWP who really put a lot of effort in at the start of our project, their expertise in WordPress was absolutely priceless for us. They were very pivotal in our project, especially in the first 12 months, and a lot of us have become really good mates.
Secondly, with the WordPress VIP team, they just kept going and going just like the energizer bunny, they never stopped no matter what challenges and issues we threw their way.
21:58
At at very high level that was our teams journey to WordPress over the last 18 months. If anyone has any questions I believe we have some time now otherwise please come find us throughout WordCamp in Vienna if you’re chasing any more details on our talk, or just want to have a chat.
Mob programming has been gathering more and more momentum over the last 6-12 months so we decided to experiment with it in some of our software development teams at NewsCorp Australia. So what exactly is Mob Programming?
”The basic concept of mob programming is simple: the entire team works as a team together on one task at the time. That is: one team – one (active) keyboard – one screen (projector of course). It’s just like doing full-team pair programming.”
Why mob programming?
How often do specific developers in a team only understand certain aspects of the code? And while code reviews are a great collaboration technique, sometimes you need more. Mob programming is a great technique to solve technical problems together as a team, allowing everyone's input to be voiced. As a positive side affect it can also help to encourage pair programming within a development team.
So how did mob programming go in NewsCorp?
Like anything you rarely find success by forcing a particular practice onto a team, it needs to adopted by a team naturally if they see it really adds value. We started by simply running a few exercises with a few of our front-end and back-end development teams to experiment on how mob programming could work within our business.
Front-end development team
Our session had 9 developers (rather large for a single session), all taking part in a JavaScript FizzBuzz challenge. Each developer was driving for a single 3 minute interval (we only had time to go around once).
Opening remarks started with, "ahh Fizzbuzz, this should be easy", and "I'm not sure how I feel about people watching me code".
Round 1: & 2:
Mood: Cautious and careful
It started off fast as the first person to drive took the laptop and started coding.. until... the team quickly pointed out, they have to listen to the navigators. This is when everything slowed down and everyone started discussing, "do we do unit tests first", "can we re-read the instructions in the readme", and "this is harder than you think when we all have to do it together". By the end of the first round, the team had created the start of the "FizzBuzz" method.
By the end of the second round the basic FizzBuzz method was complete.
Round 3:
Mood: Focused and more opinionated
Seeing the team started with the core functionality, at the start of round 3, questions were being raised about starting unit tests, and how they were going to test this functionality. By the end of round 3, we had an initial unit test that tested the basic 'FizzBuzz' was outputting correctly.
Round 4-5:
Mood: Positive and excited
Round 4 and 5 saw a wider set of unit tests being created to test the validity of Fizz, Buzz and FizzBuzz outputs.
Round 6:
Mood: Calm
There was some refactoring of the FizzBuzz application, as well as some cleanup to the unit tests to improve coverage.
Round 7-8:
Mood: Frantic and fun
By the last 2 rounds the team focused on outputting all of the numbers from 1-100 in the console. Challenge complete!
The teams observations:
The team started off slow in giving directions, well, in the way the driver understood the instructions being given. It took a few rounds for the team members to be talking the same language. After this, all someone had to say was, "created a unit test", "copy that function and past it there", "rename this to that".
As expected, there were the more confident developers navigating more than others at the start, but when it was their turn they immediately became the listeners, which gave an opportunity for other developers to voice up.
The mood was also really positive. At the start people were saying, "oh now everyone sees how I code in real time", but by the end everyone realized their peers were quite open and not judgemental.
Back-end development team:
This session had 5 developers, all taking part in the same FizzBuzz challenge (but in PHP). As before, each developer was driving for a single 3 minute interval, however we went around 2 rotations.
Opening remarks started with, "ok, lets see how this goes"
Round 1, 2 & 3:
Mood: Energetic
Team was keen to get started quickly and smashed through most of the requirements in about 2-3 rounds.
Round 4, 5 & 6:
Mood: Strategic and careful
These rounds saw the team work closely on unit testing to get confidence in the core code that was developed. A lot of discussion around correct architecture also took place, when the team decided what and how to refactor.
Round 7, 8 & 9:
Mood: Agreement, disagreement and focus
Within the last 3 rounds, the team rapidly implemented refactoring to allow better unit testing of the implementation of FizzBuzz.
Observations:
Withing 3 rounds the application was done. It was a quick MVP to get the basics in place. This development team had been working together for over 12 months so they knew how to work with each other very well.
The team took the approach of fleshing out an MVP, then adding unit tests, then refactoring the application to improve scalability. Why did they not do unit testing first? This is a team that has thousands of unit tests around their current code base, and there is a culture that code never reaches production without unit tests. To them, getting a quick MVP with the understanding that unit tests will be there for release is already well accepted.
What the team did uncover, was using sessions like this to flesh out complex features together could really speed up the overall delivery of a future feature. Currently code reviews are heavily used in this team, and this is when potential issues in design decisions are picked up. But having a 1-2 hour session to mob out the basic architecture of a new feature could really reduce feedback in code review process.
What's next? Will we keep mob programming?
This is up for each team to decide... I believe we all agree, sessions like this can be great to get an entire team over a single concept or feature, or to architect a new application with everyones input right from the start.
It was definitely a fun exercise that we will keep experimenting with.
A book for software engineering leaders who are responsible for leading teams. It aspires to provide you with helpful and reusable approaches that can be leveraged to bring about a greater level of efficiency into your role as a leader. Read more | Buy a copy
© Copyright - All Rights Reserved | Powered by Blogger
version 2.0.5