This post is co-authored by Remi Kalir and Francisco Perez.
Update 6/17/19
Highlights
- CROWDLAAERS (“crowd layers”) is a dashboard that captures open web annotation data from Hypothesis and reports learning analytics for use by learners, educators, and researchers.
- New look: We’ve streamlined the CROWDLAAERS interface to make the dashboard more intuitive, visually engaging, and easy to use.
- New features: CROWDLAAERS can now be used to access private group layers via a personal API developer token. CROWDLAAERS now also captures and reports all annotations for a given URL/document.
- New collaborative opportunity: Educators teaching with Hypothesis are invited to help pilot new Course Collections. Interested? Read on and sign up here!
Do You Annotate the Web?
Do you annotate the web? Hypothesis is a free, open-source tool that layers open annotation, conversation, and collaboration across the web. Hypothesis is regularly used by students, by scientists, and by scholars for social reading, group collaboration, and the production of knowledge. If you annotate the web – and if you’re interested in collaborative learning, open data, and learning analytics – then you’ll be eager to explore updates to the CROWDLAAERS public dashboard that provides real-time information about crowd activity for any URL annotated using Hypothesis and the various layers of annotations added to the website (for both public and now also private layers).
Launched in 2018, Capturing and Reporting Open Web Data for Learning Analytics, Annotation, and Education Researchers (CROWDLAAERS, pronounced “crowd layers”) is a dashboard that captures open web annotation data from Hypothesis and reports learning analytics for use by learners, educators, and researchers.
Curious about the annotation activity – whether public or private to a group – associated with any online article, blog post, or document? Paste a URL into CROWDLAAERS, click search, and immediately learn about who has annotated the text, when they did so, the growth of collaborative threads (i.e. replies to a initial annotation), and tags added to annotations. CROWDLAAERS quickly provides both a summary of descriptive learning analytics for a URL and also multiple interactive graphs. Dig deep into the annotation activity of an individual, explore a single thread, or use the interactive table to read individual annotations and visit the actual Hypothesis annotation in context anywhere on the web. And you can share insights with other collaborators as CROWDLAAERS also generates a public link which may be used to access the analytics of a specific website through a shareable URL.
What does CROWDLAAERS look like in action? Eager to explore collaborative learning analytics? Check out the following updates which use the annotated webpage “As We May Think” in The Atlantic, by Vannevar Bush, as an example. You can also explore the corpus of public annotation added to “As We May Think” via CROWDLAAERS.
A New Look in 2019
Our latest iteration of CROWDLAAERS features a streamlined interface that makes the dashboard more intuitive, visually engaging, and easy to use. The dashboard includes five sections: Annotations, Participants, Threads, Days, and Tags.
Each section can be expanded or collapsed (click on the section title), making it easier to focus on a particular type of analytic.
For example, here’s the expanded Participants graph that distinguishes between initial annotations (in dark blue) and replies (in dark orange) for every individual who has annotated a given URL or document. Hovering over an individual participant will display a counter summarizing their total initial annotations and total replies.
The Participants section is interactive and can be used to filter information that will appear in Annotations. For example, click on any individual in Participants and only that individual’s contributions will then appear in Annotations. These filtered results can then be further sorted by the headers in the Annotations table, including by date, tags, and level (or the position of a reply in an annotation thread). The circular arrows button at the bottom of Participants will reset the table, returning all annotations for display and no longer filtering by any individual.
Like Participants, Threads is also interactive and can be used to filter what appears in Annotations. For example, below is a single thread selected from Threads that then filters the four annotations that subsequently appear in Annotations. The Annotations table, in this image, has also been sorted by level with the initial annotation (level “0”) appearing first, followed by three replies (levels “1” and “2”).
Any individual annotation in the Annotations section can also be selected. This displays the full text of an annotation and also provides the option to view the annotation “In Context” (click the blue button). This will direct a CROWDLAAERS user back to the source URL at the precise location of the selected annotation. This is possible because every Hypothesis annotation is a distinct URL that enables direct linking.
New Technical Features
Based upon community feedback, we have added two important technical features to this latest iteration of CROWDLAAERS.
First, individual Hypothesis users can access annotations associated with all of their private group layers via a personal API developer token. Here’s a step-by-step guide to using CROWDLAAERS for access to private group annotations:
- Click the “Group layers” button in the left column
- Paste your Hypothesis API token into the field provided and click “Set token”
Don’t know where or how to access your Hypothesis API token? We have provided a link (click “Access your”) that will guide you to access your token via Hypothesis’ dashboard. When logged into Hypothesis and visiting the user dashboard, click Settings (the gear icon) and then visit the Developer section. Generate your API token (if necessary) and then copy and paste the token into CROWDLAAERS. Remember to click “Set token.”
Once your API token is added, CROWDLAAERS will allow you to sort searches according to your private Hypothesis groups. While a given URL may not have any annotations on the Public layer, toggling among private layers will provide information about the same types of analytics (Annotations, Participants, Threads, Days, and Tags) to describe activity associated with any of your private groups.
A few final and important notes regarding access to private group annotations and privacy. CROWDLAAERS does not save individual API tokens, nor does CROWDLAAERS save (in a separate database) search results associated with your private group annotations. Moreover, users do not log into CROWDLAAERS; there are no CROWDLAAERS-specific accounts that are connected to Hypothesis usernames. Your internet browser, however, may store your Hypothesis API token within the browser’s local storage. This means that after closing CROWDLAAERS in a browser window/tab and, later, reopening CROWDLAAERS, you may still have access to your private group layers. We strongly recommend that you only use CROWDLAAERS with an API token on a personal computer or clear the browser’s website data.
The second technical improvement concerns the robustness and breadth of annotations captured and reported by CROWDLAAERS. Previously, the dashboard reported only the 200 most-recent annotations. Now CROWDLAAERS captures and reports all annotations associated with a URL.
New Collaborative Opportunity: Course Collections
Many educators, and in both K12 and higher education settings, facilitate collaborative reading and discussion among their students using private Hypothesis groups. The ability to access private group activity means that CROWDLAAERS can now be used by educators as a dashboard to track course-wide and student-level annotation activity. It is also possible to track a course’s private annotation activity across multiple texts.
Educators interested in using CROWDLAAERS in their courses are invited to pilot Course Collections. A Course Collection is a course-specific CROWDLAAERS dashboard that displays annotation activity for multiple texts. A Course Collection is private. As a model, a Course Collection will be similar to the public CROWDLAAERS dashboard that reports collaborative activity associated with the Marginal Syllabus project (below).
Notice, in the image above, that the left column includes multiple links – in this case titled January, December, and Oct/November – associated with multiple texts (i.e. individual URLs), and that December is highlighted. Clicking on the name of an individual text will provide real-time analytics for annotation activity associated with the selected text/URL.
Educators who want to pilot Course Collections are encouraged to complete this form.
Upon receiving your request, the CROWDLAAERS team will then set up a private and course-specific URL similar to the URL used for all texts associated with the Marginal Syllabus project (your private URL will be similar to https://crowdlaaers.org/projects/marginalsyllabus.html). A Course Collection URL will *only* be shared with a partner educator and will not be made public. Educators can, if they chose, share their private course URL with colleagues or students. Moreover, once partner educators receive a Course Collection URL, educators will still need to add their API developer token to CROWDLAAERS for access to their private groups and students’ annotation activity.
As noted in the previous commentary about privacy, CROWDLAAERS does not save analytics associated with group activity, ensuring that students’ annotation activity (and their data) remain private. Only members of a group (whether educators or students) using CROWDLAAERS with an enabled API developer token will have access to group layer activity for the groups to which they belong.
A Final Note
CROWDLAAERS has been developed, and is currently administered, by University of Colorado Denver researchers Francisco Perez and Remi Kalir. While we partner with Hypothesis on various education and research projects, we are not Hypothesis employees and do not represent the organization. We sincerely appreciate Hypothesis’ encouragement and support in developing this public dashboard. Furthermore, we cannot guarantee that CROWDLAAERS will not be buggy. This is an experimental and academic project, not a commercial product. We welcome your feedback and appreciate your patience as we improve this public service. Please email us at info@crowdlaaers.org with any queries.
Leave a Reply