How do you measure success in the digital sphere? How should news organizations interact with their audience? What’s the best way to personalize content for individual users? These were among the topics discussed over the course of two days in February as representatives from 10 different news organizations gathered around a conference table in Austin to discuss the challenges (and opportunities) wrought by the Internet.

This particular discussion was facilitated by the Engaging News Project at the University of Texas, which brought together the journalists — which hailed from organizations like The Wall Street Journal, The Sacramento Bee, NPR, and The Texas Tribune — for a workshop on digital best practices and ideas for future experimentation.

“These sorts of conversations provide a space for organizations to work together, and I think there’s an increasing realization that for the news space to survive it’s in people’s interest to have some collaborations,” said Talia Stroud, director of the Engaging News Project.

The Engaging News Project this morning released a report summarizing the discussions which highlights various points and thoughts shared by the participants during their conversations. Here are a few highlights.

Measuring success

New technologies have allowed news organizations to tell stories in different ways online, but many still aren’t sure how to best tell a story or present information online. “How do we know whether an [interactive] infographic is better than some old-school bar chart?” Stroud asked. “This is such a profound question, right? How do we know whether the things we’re doing are working or not?” But back up: How do you even define “working”? Advertisers have their own favored audience metrics, but are they the best way to measure user engagement?

The focus is often on time on site and repeat visits, according to Tom Negrete, The Sacramento Bee’s director of innovation and news operations. (The report paraphrases the participants’ points rather than quoting them directly.) But he argues newsrooms and journalists have an obligation to go further, to measure comprehension: Can an individual understand what was just read in a news story?

To try to address this very issue, The Daily Beast has introduced a value-per-visitor metric which measures how visitors to the site read, comment, tweet, share, email, click a link, and click an app, Mike Dyer, the Daily Beast’s chief digital officer, says in the report, noting there is an economic and journalistic value to each of these actions. The Daily Beast has found, for instance, that standalone infographics are shared 300 percent more often on social than traditional articles on a similar topic. Late last year, Daily Beast staffers began meeting monthly to discuss metrics on stories, and since then monthly referrals have increased about 30 percent, Dyer said.

Measuring success is further complicated in places where there’s a traditional print or broadcast platform coexisting with digital. At The Wall Street Journal, there’s a push and pull between modes of thinking, according to Jonathan Keegan, the Journal’s director of interactive graphics. “A staffer may design a stand-alone infographic,” he’s paraphrased as saying in the report. “Copy editors may wish to hold the infographic to run alongside a news story. That is print thinking. We are getting better at realizing that graphics can go up on the site at any time.”

Improving reader engagement

Comment sections on news websites have long been derided as breeding grounds for uncivil discourse and extreme opinions. Many of the participants in the roundtable were frustrated by their comments sections and were interested in finding ways to foster more productive reader engagement.

There were various suggestions on how to reimagine comments — from inline commenting to encouraging commenters to respond to a specific question posed about the article. A consensus among the participants was that increased interaction with newsroom staffers could help with the civility dilemma — but they also acknowledged that many newsrooms do not have the resources to devote staffers to mind the comments.

Sasha Koren, The New York Times’ deputy editor of interactive news, cites the Times’ “active moderation” approach, noting that while it is heavily resource intensive, the work done to encourage meaningful comments has significant benefits for other readers.

(Still, some suggested that it’s best for reporters to stay out of the comments section. Charles Mahtesian, NPR’s politics editor for digital news, said he suggested that route because you can’t “win” against an angry commenter and dipping into the mire can be discouraging for reporters.)

How news organizations approach personalization

There was a wide disparity in how the participating news organizations thought about personalizing and segmenting their content for users, and the discussion identified seven different approaches for segmenting content: by topic, by demographics, by past site behavior, by how people come to the site, by platform, by location, and not segmenting content at all.

Stroud said she was surprised by how varied the different approaches were across news organizations, but added that all the differences could ultimately be beneficial for all news organizations. “If we want to know what works, we have to get some mechanism for assessing these sorts of things so we can distribute that information,” she said.

Again, check the full 21-page report for more about what was discussed.