14  Technical implementation

The A2SIT app is built in R, and the source code is accessible at the public A2SIT repo. The app is wrapped in an R package, and can therefore be installed locally:

remotes::install_github("UNHCR-Guatemala/A2SIT")

The front end of the A2SIT app is built using Shiny, which allows to build browser-based apps that run R behind the scenes. It also calls on many R packages for specific visualisation tasks such as mapping, plots and help popovers.

The back-end rests heavily on the COINr package which is an R package for building and analysing composite indicators. However, there is a lot of additional code which is used to build on and customise COINr functions, and to provide additional functionality. The idea is to guide the user to build their own severity index with a deliberately narrow range of options along the way. In this sense, the concept lies half way between a classical data exploration/visualisation of a single composite indicator (where the user can only explore an existing index) and a generalised GUI for building composite indicators (where a user might be able to build anything they want). This is a deliberate choice to enable an easy user interface.

The documentation here aims to be general and descriptive. Selected functions within the package are additionally documented using the usual R function documentation, and this can be accessed either using ?function_name with the package loaded, or online at the package’s documentation website. It is not possible to give an exhaustive description of the functioning of the package since it is quite complex, but hopefully the pointers here will be helpful in guiding anyone who is interested in understanding and editing the source code.

Note that the functions in the A2SIT package are highly customised for the purposes of the app, therefore even though they are accessible at the command line, many will not be hugely useful in other contexts. Probably it is more useful to use COINr directly, and the A2SIT package soruce code as examples.

14.1 Package overview

The A2SIT package is set up as a typical R package - for an overview of R package structure see the R Packages book.

fs::dir_tree("../", recurse = F)
../
├── A2SIT.Rproj
├── A2SIT_Doc
├── app.R
├── data
├── DESCRIPTION
├── dev
├── docs
├── geom_scripts
├── inst
├── LICENSE
├── LICENSE.md
├── man
├── manifest.json
├── NAMESPACE
├── NEWS.md
├── not_package
├── R
├── README.md
├── rsconnect
├── shiny_bookmarks
├── tests
└── vignettes

The main folders of interest here are:

  • /R, which contains all the R code for the front and back end of the app
  • /A2SIT_Doc, which is the source files for the long-form documentation (this book)
  • /Data, which contains data saved within the package
  • /Docs, which is the rendered version of the long-form documentation (this book)
  • /inst. which contains accessory files and resources that are used within the app
  • /man, which is the function documentation
  • /tests, which are the unit tests

The remaining folders are either created by the Golem package, which has been used to help deploy the app, or are related to smaller technical things such as Shiny bookmarking (explained later). The other files in the top level directory relate to deploying the app (app.R), and to the package itself.

14.2 Code overview

All code is found in the /R folder, as required by R package convention. The R files here aim to follow reasonably intuitive naming conventions.

To begin with, the app is run by calling the run_app() function, which lives in the run_app.R file. This is the starting point of the app, and if you want to see how the code is structured you should start here and follow the trail.

Consider that to run the app, if you have installed the package you will need to first call library(A2SIT). If you have cloned the repo (see later in this chapter) and you are working with the source code, you will need to use devtools::load_all(".") to load the functions of the package into the namespace.

At the next level down, the UI and Server functions which define the Shiny app are found in the app_server.R and app_UI.R functions respectively. If you are not familiar with the structure of a Shiny app, the Mastering Shiny book is an excellent resource.

The app is built using Shiny modules, and each module is contained within files following a mod_*.R naming pattern. The modules each correspond to the seven tabs in the app.

The modules call back-end functions which live in files following a f_*.R naming pattern. There is one of these files for each module, except the “Welcome” module/tab.

There are also a number of utility functions which are used between modules, and these live in files named as utils_*.R. The remaining files are either generated by Golem but not used very much, or relate to specific things which are mentioned later.

14.3 Back-end

A first version of the back-end code for the app was encapsulated in a standalone R package called MVI_Guatemala. This was developed before the front end of the app, and since the back end code had to be heavily modified and extended, it was simply copied into the present A2SIT package, rather than calling the previous package and needing to edit/maintain two packages. Therefore the previous package still exists as a kind of “legacy” version of the current package.

To understand the back end structure of the app, the best place to start is in app_sever.R, which defines the server function of the app. Here you will find:

  • Shared reactive variables which are passed between modules
  • The modules themselves, and variables which are explicitly passed to them
  • Some specific components that exist outside of the modules, such as export to Excel and R, the modal help windows, and the bookmarking feature

To understand each of these features, it is necessary to follow the functions called there. To give a little help, the modal help windows are written in markdown files which live in the ./inst/md-help folder, and are displayed using the “bsplus” package. Shiny bookmarking uses the “server” implementation, but additionally adds the coin to the saved file. Shiny bookmarks create folders in the ./shiny_bookmarks directory.

14.3.1 Data flow

The app is based around the COINr package, as mentioned previously. COINr encapsulates a composite indicator in an object called a “coin”, which can be manipulated by COINr functions and used to return information and plots. The coin is therefore the central object in the app.

In the input module, the user uploads a formatted spreadsheet of data and metadata, which is read by the app and used to build a coin. This task is performed by the f_data_input() function. The function reads the user spreadsheet and manipulates the data so that it can be used to create a coin. It also removes any indicators/units without data, and tries to catch any user errors in formatting and return helpful error messages.

The coin is then passed to the analysis module, which largely returns outputs to the user such as plots and tables. Here, users can optionally remove indicators, so this implies some small modifications to the coin.

Next, the coin is passed to the results module when the user arrives there. This triggers the results to be calculated, which implies running the f_build_index() function, and calculating alternative scenarios via the get_scenarios() function.

The tabs following this do not edit the coin - they simply read from it and display outputs. At any time, the coin is used to generate an Excel output using the f_export_to_excel() function, or to write the coin to an .Rdata file.

14.3.2 Mapping and geometry

It is worth saying a few particular words about maps and Admin-2 codes.

The A2SIT app aims to calculate scores at the Admin-2 level for a given country, AND to generate maps of the results. In order for this to work, it needs a geometry file for each country which defines the Admin-2 regions within that country. Moreover, each region should be referenced by a unique code which can be used within the app.

Although in theory the geometry files are available from the UNHCR’s gis.unhcr.org sever, extracting the files in the correct format has proven to be difficult. In short this is because:

  • The API tends to time out for some countries, so in practice the geometry can’t be retrieved
  • There doesn’t seem to be a harmonised system of Admin-2 codes, and codes are sometimes duplicated, which violates the requirements of COINr
  • Geometry files can be large, and some cannot be simplified.

At the time of writing, this is still an ongoing issue, and a discussion can be found on GitHub.

Leaving these pending issues aside, the A2SIT app has a function called cache_admin2_geometry() which queries the API and, if successful, stores the geometry for a given set of ISO3 codes. It also tries to simplify the geometry using the “sf” package.

The map files that are successfully retrieved are stored as .RDS files within the app’s /inst/geom folder, each of which contains a spatial polyogon data frame which can be read by leaflet. These map files are used at various points:

  1. To tell the user which countries are available for data input in the app (the dropdown menu on the “Upload” tab).
  2. To generate data input templates: the Admin-2 codes from the geometry files are used as unit codes and added to the base template. This ensures that they can be matched back to the geometry file later.
  3. To plot the maps: the user’s data is merged with the geometry file, and this enables it to be plotted.

The objective would be to have geometry files for as many countries as possible, although as mentioned this is a work in progress. Note that even for the stored geometry files, some have duplicate codes, and/or missing names.

14.4 Front-end

The app is built using Shiny, specifically using shinydashboard and shinydashboardplus. These packages give the “box” layout of the app, and also enable the sidebar menu, collapsible boxes and pop-out side bars in the boxes.

The UI of the app is defined in the app_UI.R file, where the menu items, tabs and headers/footers are specified. As mentioned previously, each tab corresponds to a Shiny module, so the UI functions can be found within the corresponding module files.

The app makes quite heavy use of add-on packages for specific functions, many of which are wrappers for JavaScript libraries, among which:

  • bsplus (modal help)
  • cicerone (guided tour on Welcome tab)
  • htmlwidgets (interactive graphics)
  • leaflet (maps)
  • plotly (interactive plots)
  • shinyjs (adding/hiding page elements)
  • shinyWidgets (enhanced user inputs)
  • webshot (exporting maps as images)

Notably, the appearance of the app is also controlled by the unhcrshiny package, which specifies the CSS.

Apart from the main UI code which is found in app_UI.R and the module files for each tab, a number of auxilliary functions are found in the utils_ui.R file.

14.5 Documentation

As an R package, the app is firstly documented in the usual way by documenting each function with a description, inputs and outputs, and so on. When the package is installed, this help is rendered and available in an R session by running the usual ?function_name command. The package doesn’t however contain any vignettes; rather everything is contained in this online book. The book is built using quarto, and the source files are available in ./A2SIT_Doc (this may be moved to ./inst at some point).

To edit the book, simply edit the source files which are written in Markdown. To render the book the easiest way is to open the book .Rproj file, which is also in ./A2SIT_Doc, in RStudio. Then, as long as quarto is installed, you should be able to click the “Render Book” button in the “Build” tab, in the upper right of the GUI. This will rebuild the entire book. Individual pages can be built by clicking the “Render” button in the file editing pane.

14.6 Contributing

The A2SIT package is hosted on a public GitHub repo at https://github.com/UNHCR-Guatemala/A2SIT.

To flag a bug, or suggest a new feature, open an issue.

If you wish to contribute code to the A2SIT package, you should:

  1. Clone the repo
  2. Create a new branch
  3. Make your edits
  4. Create a pull request

If any of these steps are unclear, a good resource for using Git and GitHub in the context of R is the Happy Git with R book. There are also countless general resources for Git and GitHub out there in the wild.