10 Reading files

As you’ve surely noticed by this point, many things in JavaScript operate on an asynchronous basis. Code is not executed linearly from beginning to end but rather in response to various triggers. For example, event listeners behave asynchronously: code will execute only if a mouse click event occurs.

The benefit to reading files asynchronously is that we don’t have to wait to while a file loads for other things to happen. It would be very frustrating to navigate to a new web page and have to wait for all the scripts to finish before we could do anything on the page.

10.1 Promises

Loading data is one area where D3 v5 introduces major changes from D3 v4. While v4 uses callbacks, v5 switches to promises, as promises facilitate cleaner and more flexile code than callbacks.

The concept is simple. We want to control what code needs to wait until data loaded to be executed and what doesn’t. We can do that with the following structure:

var rowConverter = function (d) {
  return {
    disp: +d.disp,
    mpg: +d.mpg,
    carname: d.carname,
    cylcolor: d.cylcolor
    }
};  

d3.csv("https://raw.githubusercontent.com/jtr13/d3book/master/data/mtcars.csv", rowConverter)
  .then(function(data) {

// stuff that requires the loaded data

  })
  .catch(function(error) {
  
// error handling  
  
  });

The row converter function is used to select variables and change data types (“+” converts to floating point). d3.csv() returns a promise. If the promise is resolved, the .then() function will execute; if the promise is rejected, the .catch() function will execute.

Forget the mindset that you read files and store them in variables for later use. It doesn’t work that way here. The data is read in and acted on immediately. If most of the code requires loaded data, then most of the code will appear in the .then() method.

A simple example of loading data in v5 can be found in this block. In contrast to the example above, an anonymous row converter function (with arrow functions) is used instead of calling a separate row converter function. Note as well that it’s not necessary to include all variables in the row converter as this author has done. To test, fork the block and delete all the variables that aren’t used, so that the row converter (line 56) becomes:

d => ({
            HighwayMpg: parseInt(d.HighwayMpg),
            Horsepower: parseInt(d.Horsepower),
        })

You will see that the code still works.

For more about d3.csv(), see the d3.fetch API.

10.2 Local server

For security reasons, Chrome does not let you read local files. To be able to do so, you can run a local server. One option is http-server. Follow the instructions to install http-server, navigate in a terminal to the directory with your html file, and then enter http-server. You should get a message like this:

Starting up http-server, serving ./
Available on:
  http://127.0.0.1:8080
  http://192.168.1.54:8080
Hit CTRL-C to stop the server

Copy and paste the URL in the browser and you should see your page with data loaded. As indicated, Control-c will stop the server.

10.3 Other local options

A simple way to avoid this issue is to upload data files to GitHub and read them from there. There are other workarounds, including opening Chrome from the command line with the --allow-file-access-from-files flag.

10.4 Hosting online

An alternative to the options above are to avoid the issue by hosting your code online. Options for doing so are covering in the chapter on sharing D3 online.