New Directions for the WisconsinView Data Archive

For more than 20 years the WisconsinView Data Archive (WDA) has been the go-to place for downloading free Wisconsin aerial imagery and lidar-derived elevation data products.  We are excited to announce some important positive changes affecting the archive.

Perhaps the biggest change to mention is the day-to-day management of the archive has transitioned to staff in the Robinson Map Library (RML) and State Cartographer’s Office (SCO). RML and SCO staff now have the responsibility for providing customer support, managing the contents of the archive, and growing the data collections as new lidar and imagery datasets are collected by government agencies in Wisconsin.

Previously, the UW-Madison Space Science and Engineering Center (SSEC) provided access to Wisconsin aerial imagery and lidar point clouds and lidar-derived elevation products via a public data download server. Using a desktop File Transfer Protocol (FTP) tool such as FileZilla or WinSCP, users could download large volumes of imagery and lidar data for locations throughout Wisconsin.  This included statewide imagery products produced through the Wisconsin Regional Orthoimagery Consortium (WROC), original GeoTIFF-format imagery from the National Agriculture Imagery Program (NAIP), lidar data projects initiated and paid for by city or county governments, and in recent years, lidar data collected through partnerships with federal agencies.

All of the data previously held in the SSEC FTP archive are still available, but some new tools are required for access; see below for more details.

This management change stems from a combination of factors. As the SSEC mission evolved over the years, providing access to land-based geospatial data became a lower priority for the organization. In addition, aging server hardware needed to be replaced, and staff resources associated with SSEC’s WisconsinView project were becoming more limited.  SCO and RML are well-positioned to carry on the work of SSEC, and we appreciate everything they and the WisconsinView project have done for the Wisconsin geospatial community since 2004!

Same Geoportal, New Tools for Bulk Downloads

The GeoData@Wisconsin geoportal remains in place as the main “front door” for finding Wisconsin aerial imagery and lidar data, as well as many other types of Wisconsin geospatial data.  Using GeoData@Wisconsin, you can easily search for specific spatial datasets based on criteria such as data type, location, format, and more.

The FTP site operated by our SSEC colleagues was shut down in early March of this year, and replaced with an Amazon Web Services-compatible S3 “bucket” maintained for us by the UW-Madison Division of Information Technology (DoIT).  While this does require different desktop software to access the data, you may still browse and download data just like the former FTP site.  We use the phrase “bulk download” to describe the process of browsing and downloading large volumes of data directly from the archive, versus using GeoData@Wisconsin to find and selectively download datasets for comparatively small areas.

There are some obvious advantages to providing comprehensive access to aerial imagery and lidar data along with thousands of additional geospatial data and scanned map resources in GeoData@Wisconsin. This single point of entry for access to Wisconsin data improves the user experience by making search and retrieval uniform across all data types. As longtime geospatial data managers, catalogers, and open-data proponents, this work embodies the missions of both the SCO and RML.

Want to Contribute?

We are always looking to acquire new data collections that keep GeoData@Wisconsin up-to-date and relevant for today’s data consumers.  We are happy to work with Wisconsin data producers who wish to use the WisconsinView Data Archive for data distribution.

While we do have significant holdings of imagery and lidar for all 72 counties, there are gaps.  For example, we currently do not have access to high-resolution imagery (3” and 6”) collected as part of past and present WROC projects.  Similarly, county governments often pay for and receive additional data products like 1’ contours that may not be delivered to us via FEMA or 3DEP project partners.  Designing processes and creating tools to make the data sharing process seamless for contributors (even for very large imagery and lidar datasets) has been a main focus for us in recent months.

We hope making data widely accessible through GeoData@Wisconsin means counties can spend less time and energy filling individual data requests. If you are interested in learning more, please reach out and let’s talk!

New Technology Powers the Archive

As mentioned above, our main archive consists of an S3-compatible “on premise” storage solution maintained for us by the UW-Madison enterprise storage team. Data held in S3 are redundant across three data centers in the Madison area.  Even if there were catastrophic failures at two of these data centers, data in the archive would still be preserved at a third data center.  To help guard against human error we use something called bucket versioning to keep copies of data even if they are deleted… intentionally or accidentally.

While data loss in the S3 bucket due to technology failures is highly unlikely, we still need to expect the unexpected.  Therefore, we are also using Amazon Web Services (AWS) Glacier Deep Archive storage for permanent backups of all imagery and lidar data stored in the WisconsinView Data Archive. This ensures the long-term integrity of the data, and protects us against both human-error and technology failures in the main S3 archive.

As we write this, the total WisconsinView Data Archive is just over 90 Terabytes in size.  Thanks to our UW-Madison affiliation, we are fortunate to have the network bandwidth, transfer tools, and the added benefit of no egress (data download) charges that allow us to move around this much data and easily make it publicly accessible!

Contact Us

Please contact us if you have questions about downloading data, searching GeoData@Wisconsin, or are interested in contributing your own geospatial data to the WisconsinView Data Archive.