One of the key things we must avoid is mess and confusion. In the last post I briefly covered one possible template for a research project.
One of the reasons why cognitive modelling projects might be a little tricky is because you are not just using an off the shelf software package. For example, if you were doing some data analysis then you might write a script (utilising an existing toolbox/package) to load your data, transform it, conduct some analysis, then visualise the output.
But in cognitive modelling, as well as using existing packages you are actively developing your own modelling code as well as writing a script that uses it. In the past I have forgotten that these are two separate (yet interacting) things. My initial, bad, approach was to have a big jumbled folder of code, and then I manually conducted a series of magical steps which were only written down in my head. Coming back from a break caused me to forget what order to call functions, or what arguments they needed etc. It was not fun.
One way how the simple folder structure helps is by explicitly keeping all the complex model code you write separate from the all important script. The script is crucial in that it is a readable record of what you did and why. My simple recommendation here is that we explicitly realise the importance of separating the ideas of:
- building, changing, perfecting cognitive model code, and
- creating a script (like a lab notebook) that people can follow in order to understand the steps we have made and the claims we make in a research paper.
iPython / Jupyter as the script
I am still transitioning away from Matlab and towards Python, but one big appeal is the iPython project. This started out as a really interesting way to create a kind of live lab notebook (or script) which can contain documentation, runnable code, figures, even embedded youtube videos, all in one place. This started out being focussed upon Python, but Jupyter extends this to multiple languages including R, Julia, etc. There are many awesome introductions to the approach, and I’ve included one below which emphasises its role in reproducible research. We can create a simple script using whatever programming language we may want, but I suspect that the iPython (jupyter) approach will be the way of the future. Find out more from this write-up in Nature.