Snapshot Wisconsin relies on different sources to help classify our growing dataset of more than 27 million photos, including our trail camera hosts, Zooniverse volunteers and experts at Wisconsin DNR. With all these different sources, we need ways to assess the quality and accuracy of the data before it’s put into the hands of decision makers.
A recent publication in Ecological Applications by Clare et. al (2019) looked at the issue of maintaining quality in “big data” by examining Snapshot Wisconsin images. The information from the study was used to develop a model that will help us predict which photos are most likely to contain classification errors. Because Snapshot-specific data were used in this study, we can now use these findings to decide which data to accept as final and which images would be best to go through expert review.
Perhaps most importantly, this framework allows us to be transparent with data users by providing specific metrics on the accuracy of our dataset. These confidence measures can be considered when using the data as input for models, when choosing research questions, and when interpreting the data for use in management decision making.
The study examined nearly 20,000 images classified on the crowdsourcing platform, Zooniverse. Classifications for each specie were analyzed to identify the false-negative error probability (the likelihood that a species is indicated as not present when it is) and the false-positive error probability (the likelihood that a species is indicated as present when it is not).
The authors found that classifications were 93% correct overall, but the rate of accuracy varied widely by species. This has major implications for wildlife management, where data are analyzed and decisions are made on a species-by-species basis. The graphs below show how variable the false-positive and false-negative probabilities were for each species, with the whiskers representing 95% confidence intervals.
Errors by species
We can conclude from these graphs that each species has a different set of considerations regarding these two errors. For example, deer and turkeys both have low false-negative and false-positive error rates, meaning that classifiers are good at correctly identifying these species and few are missed. Elk photos do not exhibit the same trends.
When a classifier identifies an elk in a photo, it is almost always an elk, but there are a fair number of photos of elk that are classified as some other species. For blank photos, the errors go in the opposite direction: if a photo is classified as blank, there is a ~25% probability that there is an animal in the photo, but there are very few blank photos that are incorrectly classified as having an animal in them.
Assessing species classifications with these two types of errors in mind helps us understand what we need to consider when determining final classifications of the data and its use for wildlife decision support.
When tested, the model was successful in identifying 97% of misclassified images. Factors considered in the development of the model included: differences in camera placement between sites; the way in which Zooniverse users interacted with the images; and more.
In general, the higher the proportion of users that agreed on the identity of the animal in the image, the greater the likelihood it was correct. Even seasonality was useful in evaluating accuracy for some species – snowshoe hares were found to be easily confused with cottontail rabbits in the summertime, when they both sport brown pelage.
Not only does the information derived from this study have major implications for Snapshot Wisconsin, the framework for determining and remediating data quality presented in this article can benefit a broad range of big-data projects.
In late February this year, a Snapshot Wisconsin trail camera deployed in Vilas County captured an American marten (Martes americana). This is the first time an American marten has been captured on a Snapshot Wisconsin camera! The below American marten was identified by the trail camera host, Ashley, and the identification was then confirmed by several species experts in the Wisconsin DNR. While American marten can vary in color, they are best identified by their pale buff to orange throats, dark legs and tails, vertical black lines running above the inner corners of their eyes, and bushy tails that account for one-third of their total length.
Extirpated from Wisconsin in the 1940’s, these small members of the weasel family were later reintroduced to the state and placed on the Wisconsin Endangered Species List in 1972 due to loss of suitable habitat. Marten are restricted to the northern portion of the state where they reside in dense, mature forests with preference for areas that are a mix of coniferous and deciduous trees.
Did you know that marten are excellent climbers? They use this skill not only to hunt down prey, but also to avoid potential danger. These solitary animals are very territorial, with territories spanning an average of two square miles for males and one square mile for females. Although the breeding season lasts from July to August, fertilized eggs do not fasten to the uterine wall until January or February. Females birth two to four kits in March or April, and raise their young in tree dens without any male assistance.
There is still much to be learned about American marten, as their nocturnal lifestyle and often shy demeanor make them a difficult species to study. Follow this link for more information about American marten in Wisconsin, and stay tuned to discover what rare species will be captured next on Snapshot Wisconsin trail cameras!
The Snapshot Wisconsin team is often asked why we accept data only from our Snapshot-specific cameras. While there are several reasons, the reason that was highlighted in the April 2019 newsletter was because Snapshot Wisconsin cameras are programmed to take a single photo at 10:40 a.m. each day. Although 10:40 may seem like an arbitrary time, this corresponds to the approximate time that a NASA satellite flies over Wisconsin and collects aerial imagery. (More information on how NASA data and Snapshot data are complementary can be found in this blog post.)
It may be difficult to recognize the value of a blank photo in wildlife research, but a year-long series of these photos allows us to examine something very important to wildlife: habitat condition. For each camera site, the time-lapse photos are loaded into the statistical software, “R,” where each pixel in the image is analyzed and an overall measure of greenness is summarized for the entire photo. That measure, called the Green Chromatic Coordinate, can be used to identify different “phenophases,” or significant stages in the yearly cycle of a location’s plants and animals. These stages can be delineated on a graph, called a phenoplot, where a fitted curve reveals the transition day-by-day. The 2018 phenoplot for one Snapshot Wisconsin camera site is seen below.
In 2018, 45 camera sites had a complete set of 365 time-lapse photos, but we expect many more sites to be included in the 2019 analyses. The relatively small sample size for 2018 is due in part to many counties not being opened for applications until partway through the year, but also because time-lapse data are rendered unusable if the date and time are not set properly on the camera. This may happen when the operator accidentally sets the time on the 12-hour clock instead of the 24-hour clock, or if the hardware malfunctions and resets the date and time to manufacturer settings—this is why we ask our volunteers to verify the camera’s date and time settings before leaving the site each time they perform a camera check.
The information derived from these analyses will be integrated into wildlife models. For example, the objective of one ongoing DNR research project is to understand linkages between deer body condition and habitat, which includes what’s available to deer as forest cover and food resources, as well as weather-related factors, such as winter severity or timing of spring greenup. The project currently uses weather data collected across the state to estimate snow depth, temperature, and winter severity, and creates maps based off this information.
Snapshot’s time-lapse cameras offer a wealth of seasonal information regarding type of forest cover and food sources, as well as weather-related information. In the future, phenological data obtained from Snapshot cameras could be used to create “greenup maps” that provide estimates of where and when greenup is occurring, and potentially test that information as a means of better understanding how environmental factors affect deer health, such as whether an early spring greenup improved deer body condition the next fall.
One of Snapshot Wisconsin’s major goals is to alleviate some of the burden associated with time-consuming in-person survey techniques. This is possible because trail cameras can serve as round-the-clock observers in all weather conditions. Annual Greater Prairie-Chicken lekking (breeding) surveys were identified as having good potential to be supplemented by Snapshot Wisconsin cameras, and a pilot study was conducted in spring 2018.
The Greater Prairie-Chicken (GPC) is a large grouse species native to grassland regions of central Wisconsin. During the breeding season each spring, males compete for female attention by creating a booming noise and displaying their specialized feathers and air sacks. This ritual occurs on patches of land known as leks, as seen in the photo above. Wisconsin DNR Wildlife Management staff identify leks in the early spring and return to each site twice in the season to count the number of booming males. The number of males present on the leks is used as an index to population size. Three Snapshot Wisconsin cameras were deployed on each of five leks – one camera facing each direction except for east to reduce the number of photos triggered by the rising sun. The cameras were deployed from late March through mid-May, and all in-person surveying was conducted within the same period.
As seen in the graph above, Snapshot Wisconsin trail cameras recorded male GPC at all five of the study sites. This is significant because GPC were only detected on three of the five leks according to the in-person surveys. On leks A, B, and D, where both in-person and camera surveying detected GPC, the in-person maximum of male GPC was higher. However, when the trail camera maximum is averaged across all survey days, the maximum is nearly the same for both survey methods (8.5 in-person, 8.3 trail camera).
In-person surveying requires the observers to arrive before dawn and remain in the blind until after the early morning booming has finished. Snapshot Wisconsin cameras record the hourly activity on the lek while minimizing the risk of disturbance due to human presence. The graph above displays the total number of male GPC photos captured by hour and shows a small uptick in photos around 7 p.m. Because the in-person surveys do not include evening observations, Snapshot Wisconsin data offer a way to examine the lek activity at all hours.
Additionally, continuous data collection is not only useful in capturing the activity of GPC, but offers insight into the dynamics of Wisconsin’s grassland ecosystems. In total, Snapshot Wisconsin cameras collected over 3,000 animal images including badger, coyote, deer, other bird species, and more. Some photos were even a little surprising. Pictured above is a coyote just feet away from prairie chicken. We might expect the GPC to flee in the presence of a predator, but this one appears to be standing its ground. In the upcoming pilot year two, we hope to gather even more information about the interactions within and among species found on these leks.
One of the major Wildlife Management implications for Snapshot Wisconsin is the project’s contributions toward a system the DNR uses to calculate the size of the white-tail deer population in Wisconsin. Fawn-to-doe ratios, or FDRs, are found by dividing the number of does by the number of fawns seen during the summer months and are summarized by the (82) management units across the state.
In total, three programs contribute to FDR estimates: Snapshot Wisconsin, Operation Deer Watch, and the Summer Deer Observation Survey. An advantage of incorporating Snapshot Wisconsin data in these estimates is that Snapshot cameras tend to be placed in secluded, natural areas, whereas the other two collection methods are opportunistic, meaning they’re biased toward counting deer seen near roadways.
One challenge associated with trail camera data is that the same individual animals may walk by the camera multiple times throughout the data collection period. To account for this, we average the total number of does seen in photos with at least one doe, and then average the total number of fawns in each photo containing at least one fawn. We then take the average number of fawns and divide it by the average number of does.
Fawns and does may or may not be in the same photo to contribute to their respective averages. Defining a single camera-level average for each site drastically reduces the amount of data involved but ensures that the FDR is not skewed toward does, which tend to appear much more frequently on Snapshot cameras.
The above maps show the camera sites that contributed to FDR estimates in 2017 and in 2018. Photos from exclusively July and August were analyzed. A site only contributes to the estimate if there were at least 10 doe observations in one of the two months, but can be counted twice if it had at least 10 doe observations in both months. Statewide, 897 cameras contributed to 2018 FDR estimates, a 44% increase from the 622 sites that contributed in 2017. Some deer management units decreased in sample size from 2017, but
Above are the results of the 2017 and 2018 FDR estimates using Snapshot Wisconsin data. Only deer management units with a minimum of 5 camera sites were included in the analysis. In 2018, the range of FDR was 0.75 – 1.2, which is an overall increase from the range of 0.62 – 1.13 in 2017. Snapshot Wisconsin was launched statewide in August 2018, meaning most cameras in the newly open counties were not deployed until after the data collection period. We expect that the number of cameras in the 2019 analysis will increase again, which would give us even more accurate estimates.
For January’s Science Update, also featured in The Snapshot monthly e-newsletter, we explored the accumulation of Snapshot Wisconsin photos over time and how the number of photos taken fluctuates with the seasons. To date, our data set contains more than 24 million photos, and their content is a vital component of the Snapshot Wisconsin project.
The bar chart above indicates that over half of the photos are blank. This can be attributed to the fact that our cameras contain a motion trigger function, which is designed to capture wildlife as it moves through the frame. However, this mechanism only detects movement and cannot differentiate between animals and vegetation. This means that on windy days during the spring green up period, thousands of blank photos can be captured. Occasionally cameras will malfunction and continuously take blank photos without being triggered by motion. This issue was more prevalent with earlier versions of our cameras; the model we currently use does not take as many blank photos. Additionally, over time volunteers have learned that trimming vegetation in front of their camera helps prevent blank photos.
Every day at 10:40AM, the cameras are programmed to record a time lapse photo. This is not only to document the “spring green up” period and the “fall brown down” period, but also to sync ground-level measures of greenness with satellite data. These photos are primarily used by our partners at UW-Madison and compose 7% of our data set.
It is not uncommon for our trail camera hosts to trigger the camera themselves during check events, which is the cause of most of the 3% of photos that are tagged as human. Although these photos are removed from the data set prior to analysis, they can be helpful in instances where the camera has been recording photos with the wrong date and time. A photo of a hand in front of the camera combined with the date and time reported by the volunteer at each check event are enough for us to adjust the date and time for the whole set of photos.
Twenty percent of the Snapshot Wisconsin photos are untagged, meaning they have yet to be classified as blank, human or animal. Many of these photos will be sent to the crowd sourcing website, Zooniverse, for classification. We hope to implement a program to automatically classify photos to work through this backlog as well.
Finally, about 14% of Snapshot Wisconsin photos are of confirmed animals. In the graph above, we have broken down which species appear in these photos. Deer are by far the most common species, appearing in about two-thirds of photos, followed by squirrels, raccoons, turkey, cottontail rabbits, coyotes, and elk. The remaining 8 percent of animal tags are divided up across 34 categories including other bird, opossum, snowshoe hare, bear, crane, and fox. Elk may have a higher proportion of triggers than expected because Snapshot Wisconsin cameras are placed more densely in the elk reintroduction areas than in other areas of the state.
As many volunteers may be aware, Snapshot Wisconsin operates continuously and year-round. This is distinct from many other studies and monitoring efforts focused upon wildlife that typically evaluate population status during a particular time of year and look at changes between years. One belief that underpins Snapshot Wisconsin is that because the environment consistently changes across the year, how animals move, behave, live, die also changes. Consistent and continuous data collection provides the project much richer insights into animal habitat associations, and it also gives the project the ability to evaluate which time of year deserves most focus–when should we be monitoring things?
As discussed in previous posts, we heavily rely on spatial data produced by processing satellite imagery in order to quantify species’ habitats and estimate or predict where animals are. Recall that two important sensors or satellites are Landsat, which has fine spatial resolution but captures imagery of Wisconsin less frequently, and Modis (on the Aqua and Terra Satellites) that has coarser spatial resolution, but captures an image of Wisconsin daily. Data produced from Modis imagery is incredibly useful for capturing, say, the timing of larger scale phenomena like big snow events, or the onset of either long-term snow coverage in the winter or green up in the spring. The behaviors and activity of animal species are often connected to the timing of these environmental cues (or others, like temperature, or the length of daylight).
One animal species that is particularly sensitive to seasonality is the black bear: bears spend the winter in dens in a state of torpor. In brown bears, previous research has suggested that the timing of bear den entry is sensitive to environmental factors, while the timing of ending torpor is more related to individual physiology. One thing we are interested in is whether bear behavior (out and about, or in torpor) exhibits any correlation with the variation in the timing of plant green-up and senescence across space, and whether “mismatch” between when bears exit dens and when plants green up (plants like sedges are an important food source for bears early in the year) seems to have any population consequences.
Snapshot Wisconsin cameras capture bears growing and moving across seasons.
While Wisconsin vegetation greens up from late winter to mid-summer, bears also become increasingly present across the landscape.
BUT…..while satellite imagery provides an excellent overhead synopsis of plant activity, it is not always clear which plants are making the image green. Bears–and many other animal species–primarily eat green matter at ground level rather than leaves at the tops of trees. Snapshot Wisconsin’s cameras provide a ground view that we can relate to satellite images to get a sense of what airborne imagery is responding to. In the long term, this will allow improved estimates of where animals are and at what time.
As graduate student on the Snapshot Wisconsin project, part of my role is to help the team better understand their volunteers and conduct research that will assist with program improvement. One way I do this is by surveying trail camera hosts when they enter the program and after they have been participating in Snapshot Wisconsin for one year.
Developing a survey takes more work than you might expect! Some things, like age or occupation, are relatively easy to measure. However, abstract concepts like satisfaction or attitudes are much more difficult to capture in a survey. These abstract concepts must be measured in more indirect ways, and typically social scientists develop a number of survey questions or items to measure a concept.
For example, let’s say I wanted to measure someone’s job satisfaction through a survey. You could ask, “How happy are you overall with your job?” (Rate 1-5).
In order to capture more aspects of job satisfaction, it would be better to ask: “How happy are you with each of the following parts of your job? Autonomy, work load, salary, coworker relations, etc.” (Rate each 1-5).
Bear with me while I get theoretical for a moment…
Imagine you have a whole universe of survey items you could ask someone about job satisfaction. If you choose just one question to ask them, that question is not likely to be a good representation of their job satisfaction as a whole. However, if you ask them multiple questions, you get a much better representation of their job satisfaction.
Let me use an analogy. If I want to know all the different species of mammals found in a particular county and I put out just one trail camera in that county, it isn’t likely to be sufficient. I put out a whole bunch of cameras across the county, I’d get a much more accurate count.
Often, I get this question from people who take surveys: Why do some of these survey questions seem so similar to one another? Can’t you ask this with just one question?
The answer is: if we are asking about an abstract concept in a survey, assessing it indirectly though multiple questions is the best way to go if we want valid scientific results.
Through email and the internet it is so easy to deliver surveys and if you are like me, you get a survey in your inbox from some business or organization just about every week. Hopefully this sheds a little light on what goes on behind the scenes before you get that “new mail” notification.
For those of you who have completed a Snapshot Wisconsin survey, your responses are truly valued. We are learning a lot; see here for some early results and keep your eyes on the blog for more. If you are interested in learning more about the science behind surveys, let me know in the comments!
One of the objectives of Snapshot Wisconsin is to record the occurrence of rare species including: moose, cougar, Canada lynx, marten, jackrabbit, Whooping crane, Spotted skunk, and wolverine. With a statewide network of nearly 1,300 trail cameras, sooner or later we were bound to capture one of these rare Wisconsin species. Two years into the project, Snapshot Wisconsin captured its first – moose (Alces alces)!
Earlier this month, we received an email from a volunteer in Oneida county with the subject ‘Picture of Moose.’ We nearly jumped out of our seats exclaiming “Moose! Moose! Moose!”
From the size and proportions of the animal, it was easy to tell that it was indeed moose. Moose can reach upwards of 1,500 pounds and stand up to 7 feet tall, dwarfing our commonly seen White-tailed deer. When we shared the picture around, our Wildlife Research team leader remarked, “That part of the state is definitely moose-y.” The bogs of Oneida, Vilas, and Iron counties have had the most moose sightings in the recent years, making “moose-y” an apt description.
Upon querying our Snapshot Wisconsin database, we found another moose identified on a camera in Vilas county – this one hosted by an educator. Both of these sightings were from spring this year, and both were correctly identified by the volunteer – hurray, no ‘moose-takes’ there!
Moose are categorized as a species of special concern in Wisconsin due to their relatively low numbers, in 2016 there were only 32 possible or probable observations reported.
Whether you are a Zooniverse volunteer or a trail camera host, please let us know if you see a rare species in a Snapshot Wisconsin photo. If you spot them in the wild or on a personal trail camera, report the observation using the Wisconsin large mammal observation form. In the meantime, we hope you finding these pictures as ‘a-moos-ing’ as we do!
What looks like a chicken, lives in the prairie and has one of the most phenomenal displays of courtship in the animal kingdom? You guessed it, the greater prairie chicken (Tympanuchus cupido)! Earlier this spring, Snapshot Wisconsin teamed up with WDNR biologists in the Buena Vista Grasslands area to implement a prairie chicken trail camera monitoring project in Wisconsin.
In the 1950’s, the greater prairie chicken was close to extinction in the state of Wisconsin. The WDNR, in partnership with conservation groups, established a prairie chicken management program. Every year in early spring, WDNR biologists begin thoroughly surveying known greater prairie chicken lekking grounds to track population size and locations of leks. The protection and monitoring of the species has helped the comeback of the prairie chicken in Wisconsin. Currently, a few thousand chickens can be found in the central part of the state.
This year, Snapshot Wisconsin deployed 15 cameras to help supplement monitoring efforts. Trail cameras can efficiently and continuously survey known lekking grounds to count peak numbers of males at each lek.
The team had a blast deploying trail cameras at lekking grounds. For those of you who own trail cameras, you may be familiar with the walk test. In the chicken camera test, we ended up crawling to test if a chicken would be detected on camera. This resulted in a lot grass in our clothes and an equal amount of laughter.
For more information on Wisconsin’s prairie chickens check out this link.