10 Reading files

As you’ve surely noticed by this point, many things in JavaScript operate on an asynchronous basis. Code is not executed linearly from beginning to end but rather in response to various triggers. For example, event listeners behave asynchronously: code will execute only if a mouse click event occurs.

The benefit to reading files asynchronously is that we don’t have to wait to while a file loads for other things to happen. It would be very frustrating to navigate to a new web page and have to wait for all the scripts to finish before we could do anything on the page.

10.1 Promises

Loading data is one area where D3 v5 introduces major changes from D3 v4. While v4 uses callbacks, v5 switches to promises, as promises facilitate cleaner and more flexile code than callbacks.

The concept is simple. We want to control what code needs to wait until data loaded to be executed and what doesn’t. We can do that with the following structure:

const rowConverter = function (d) {
  return {
    disp: +d.disp,
    mpg: +d.mpg,
    carname: d.carname,
    cylcolor: d.cylcolor
    }
};  

d3.csv("https://raw.githubusercontent.com/jtr13/d3book/main/data/mtcars.csv", rowConverter)
  .then(function(data) {

// stuff that requires the loaded data

  })
  .catch(function(error) {
  
// error handling  
  
  });

The row converter function is used to select variables and change data types (“+” converts to floating point). d3.csv() returns a promise. If the promise is resolved, the .then() function will execute; if the promise is rejected, the .catch() function will execute.

Forget the mindset that you read files and store them in variables for later use. It doesn’t work that way here. The data is read in and acted on immediately. If most of the code requires loaded data, then most of the code will appear in the .then() method.

A simple example of loading data in v5 can be found in this block. In contrast to the example above, an anonymous row converter function (with arrow functions) is used instead of calling a separate row converter function. Note as well that it’s not necessary to include all variables in the row converter as this author has done. For example, you could delete all the variables that aren’t used, so that the row converter in the d3.csv line becomes:

d => ({
            HighwayMpg: parseInt(d.HighwayMpg),
            Horsepower: parseInt(d.Horsepower),
        })

You will see that the code still works.

For more about d3.csv(), see the d3.fetch API.

10.2 Local server

Note: As long as you read from online sources as in the example above, you do not need to set up a local server.

For security reasons, Chrome does not let you read local files. To be able to do so, you can run a local server. One option is http-server. Follow the instructions to install http-server, navigate in a terminal to the directory with your html file, and then enter http-server:

joycerobbins@MacBook-Pro d3-book-murray % http-server

You should get a message that ends with something like this:

Starting up http-server, serving ./
...
Available on:
  http://127.0.0.1:8080
  http://192.168.1.54:8080
Hit CTRL-C to stop the server

Copy and paste one of the URLs in the browser and you should see an index of subfolders and/or files available in the folder in which you launched the local server:

From here you can navigate to the desired file. Take note that you cannot move up in the file structure so be sure to start the server in the highest level directory that you plan to access, or that the files you open need to access. In this particular case I am opening the code files for Scott Murray’s Interactive Data Visualization for the Web, 2nd ed. (available here). Rather than link to the online version of D3 as we’ve been doing, each file links to a downloaded version of D3 located in the top level directory with:

<script type="text/javascript" src="../d3.js"></script>

Therefore we must launch the local server from one level higher than the folder in which the file we wish to open resides.

As indicated, Control-c in the command line will stop the server.

10.3 Other local options

A simple way to avoid this issue is to upload data files to GitHub and read them from there. There are other workarounds, including opening Chrome from the command line with the --allow-file-access-from-files flag.

10.4 Hosting online

An alternative to the options above are to avoid the issue by hosting your code online. Options for doing so are covering in the chapter on sharing D3 online.