With the recent release of Houdini 21, I have been looking into the Houdini Machine Learning tools for making regression models. From my research I made a model based off a FLIP simulation: Artstation Post. With this blog, I hope to give enough information to Houdini user of the core steps to making a basic regression model.

So what is Machine Learning? As defined by IBM, “Machine learning is the subset of artificial intelligence (AI) focused on algorithms that can ‘learn’ the patterns of training data and, subsequently, make accurate inferences about new data.” In Houdini, we make this algorithm by creating examples with the ML_Example SOPs node, then training a network with the ML_TrainRegression TOPs node. These are the core of ML model making in Houdini.

ML_Example (SOPs): Each example needs two components: the first being the input component (this is your model’s initial shape) and the second being the target component, what your model is aiming to make. In my fluid model, the examples are made in a for each point loop where I use the current frame as the input component and the next frame as the target component. 

That being said, the components of an example do not need to have the same attributes or be the same type of geometry (however, the attributes must be floats). In some of the other ML projects shared on the SideFX website, we can see the model solving geometry deformation (target) from a rig position (input) (SideFX Link). 

The output of this node is packed primitives, these are your examples for your model training. If you want to do further processing, there is an ML_ExampleDecompose node which will split them so you can alter them and then put them back together with ML_Example again. 

Next, we will be taking our examples to TOPs for training. To do this, we will use ROP_ML_ExampleRaw node. In the parameters, we need to specify the attributes that will be used for training in our input and targets. For example, if you want the P attributes from the input components to be used to determine what the Cd and N values should be in the target components, then you will fill it in like this:

After you’ve rendered this node, you will have a data_set.raw file in your $HIP file that will be used to train the model in TOPs.

ML_TrainRegression (TOPs): To train the model, you can simply place the ML_TrainRegression node down in a TOPs network and cook it (as long as you didn’t change the data_set.raw file name). To get better results, though, you will need to adjust the parameters based on your examples and goals. In the Network>Model node folder, you can adjust the ‘Hidden Layers’ and ‘Hidden Width’. Here I have made a graph based off the default hidden layers and hidden width of the training node:

This is a neural network. On the left, we can see the input attributes I’ve used @P for the input and output as an example. Each of the @P values gets passed through each of the 256 neurons (labelled n# in the graph) in the 0th layer of the network. As you can infer, the Hidden Width parameter is referring to the number of neurons per layer. Each neuron contains a 'weight', the value of the @P is multiplied by this weight, with the output passing onto the next layer (the 1st layer in this graph), where it is multiplied again by another set of weights, then passed along to the output attributes that we specified. These weights are adjusted during training until the model is able to get the correct output. Again, it can be inferred that the Hidden Layers is referring to the number of layers in a network.

So what should you set your Hidden Layers and Hidden Width to? Ultimately, this needs to be decided through trial and error (yay!). However, as a rule of thumb, start out with a small size such as the default settings and build up the model depending on the output. As the size of the model gets bigger, you will need more examples. With too big of a model and too few examples, you risk overfitting the model (the model will only work well for the specific situation from your training data). So how many examples do you need? Once again, there is no one-size-fits-all. There are many debates I have read through that have vastly different opinions, with some saying to just ‘follow intuition’. But for someone like myself who has never worked with ML models before and has no intuition, I tried to have 0.5 to 1 examples per parameter in the neural network. Parameters in neural networks can be calculated by: (Number of inputs × width of 0th layer) + (width of 0th layer × width of 1st layer) + (width of 1st layer × number of outputs). This is excluding biases, but for our sake, it’s not necessary to have an exact number.

 So, for the network from my graph, I would have aimed to have 30k/60k examples. Again, this number would be dependent on the model that you’re creating, you may only need 16k examples for your model to work sufficiently so it's best to test out different data set sizes.

The Weight Decay parameter (Network > Regularizer > Weight Decay) can be used to reduce the overfitting of your model. The weight decay stops the weights in each neuron from getting too big and, in turn, memorizing the input data rather than learning concepts. If your outputed model isn’t working with alternative inputs, experiment with setting this a bit higher (my model ended up using 0.0001).

Learning Rate (Training > Optimization > Learning Rate): this adjusts how quickly the model changes its weights. For my model, I didn’t use this, as I found through trial and error that the default fitted the best. However, if you find that your model is overfitted, a lower learning rate may be better, as it is learning too much from a single example. If the model isn’t doing much, it may need a higher learning rate, as it isn’t able to learn any key details from your examples.

For every pass through all your training data (each full pass is called an epoch), the model will calculate its accuracy and then adjust the weights (the values in each of the neurons). The model finishes training after it stops improving this accuracy score. If you want to adjust how long it will keep trying to improve this score, change the value of the Patience parameter (Training > Termination > Patience). Less Patience will result in a less accurate model, however, if your model is too patient, you may be waiting for a very long time. To see the accuracy score during training, click on the work item for the training node while training.

ML_RegressionInference: Once you have trained your model, we can go back to SOPs and use the ML_RegressionInference node. This node is straightforward: select the model file, set the input and output attributes the same as you set them in the ROP_output node, with the only difference being if your output attribute is a vector, you need to specify the size (@P would be set to 3). What you plug into the model needs to have the same attributes and same geometry size(point number) as what you plugged into the ML_Examples node. For mine, I placed it inside a forEach point loop.

If everything has worked out well, you should have a pretty basic model. Further development would include looking into Principal Component Analysis for data optimization (PCA Dejitter, ML for VFX Artist).

Final thoughts

While it is easy to get excited about the uses of machine learning in Houdini, not every use case makes sense. Unless the model is being made to increase optimization and speed, using Houdini to create thousands of examples inherently defeats the purpose of ML. Problems best suited for a ML algorithm is one where a typical algorithm is not easily made, such as recognizing a cat in an image. With this in mind using Houdini to generate data for a model defeats the purpose, as we already had an algorithm to make that data. Using Houdini to process human made data for a model would be where I am most interested to see progress. For example in Dune 2, they were able to use the manually rotoscoped eyes from the first film to train a model to make this proccess quicker for the second film. With that being said these examples need to be ethically sourced and not removing the autonomy of the artist, who, should be our highest priority when making these tools. 

In short when deciding if a problem should be solved with Machine Learning we have to consider how we are going to collect the data needed and whether it would be easier and more accurate to use good old fashion coding. With cases that are labour heavy, such as art heavy tasks, we also need to need to remember that the tools we make need to aid the artist to be more creative rather than attempt to replace them.

Thank you for reading this post! I hope it is informative and helpful in your journey into Machine Learning.