{"id":3901,"date":"2016-05-19T14:46:56","date_gmt":"2016-05-19T12:46:56","guid":{"rendered":"https:\/\/blogs.uni-paderborn.de\/sse\/?p=3901"},"modified":"2016-05-19T14:47:32","modified_gmt":"2016-05-19T12:47:32","slug":"what-to-do-with-rejected-artifacts","status":"publish","type":"post","link":"https:\/\/blogs.uni-paderborn.de\/sse\/2016\/05\/19\/what-to-do-with-rejected-artifacts\/","title":{"rendered":"Should we reject papers with bogus artifacts?"},"content":{"rendered":"<div class=\"twoclick_social_bookmarks_post_3901 social_share_privacy clearfix 1.6.4 locale-en_US sprite-en_US\"><\/div><div class=\"twoclick-js\"><script type=\"text\/javascript\">\/* <![CDATA[ *\/\njQuery(document).ready(function($){if($('.twoclick_social_bookmarks_post_3901')){$('.twoclick_social_bookmarks_post_3901').socialSharePrivacy({\"txt_help\":\"Wenn Sie diese Felder durch einen Klick aktivieren, werden Informationen an Facebook, Twitter, Flattr, Xing, t3n, LinkedIn, Pinterest oder Google eventuell ins Ausland \\u00fcbertragen und unter Umst\\u00e4nden auch dort gespeichert. N\\u00e4heres erfahren Sie durch einen Klick auf das <em>i<\\\/em>.\",\"settings_perma\":\"Dauerhaft aktivieren und Daten\\u00fcber-tragung zustimmen:\",\"info_link\":\"http:\\\/\\\/www.heise.de\\\/ct\\\/artikel\\\/2-Klicks-fuer-mehr-Datenschutz-1333879.html\",\"uri\":\"https:\\\/\\\/blogs.uni-paderborn.de\\\/sse\\\/2016\\\/05\\\/19\\\/what-to-do-with-rejected-artifacts\\\/\",\"post_id\":3901,\"post_title_referrer_track\":\"Should+we+reject+papers+with+bogus+artifacts%3F\",\"display_infobox\":\"on\"});}});\n\/* ]]> *\/<\/script><\/div><p>Yesterday I blogged about the <a href=\"https:\/\/issta2016.cispa.saarland\/accepted-artifacts\/\">accepted artifacts at ISSTA.<\/a>\u00a0I think it is worthwhile noting that out of the ten papers that got accepted and which had artifacts submitted there\u00a0seven for which the artifacts checked out. That is a good thing!<\/p>\n<p><strong>What worries me, however, are the three papers for which Artifact evaluation failed. For those three papers, we were largely unable to reproduce their results, and yet the papers made it into the program.<\/strong> Moreover, for two of those three, the fact that they failed artifact evaluation was already known before the PC meeting, i.e., there would have been a chance to reject them.<\/p>\n<p>The reason for why the PC\u00a0did not is that only positive reviews were taken into account this time, in order not to discourage people from submitting artifacts in the future. We as a community should really think about whether we cannot find a way to make artifact evaluation the default so that people have no other chance than to submit all the evidence they have to back up their claims.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Yesterday I blogged about the accepted artifacts at ISSTA.\u00a0I think it is worthwhile noting that out of the ten papers that got accepted and which had artifacts submitted there\u00a0seven for which the artifacts checked out. That is a good thing! &hellip; <a href=\"https:\/\/blogs.uni-paderborn.de\/sse\/2016\/05\/19\/what-to-do-with-rejected-artifacts\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":6542,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3901","post","type-post","status-publish","format-standard","hentry","category-general"],"_links":{"self":[{"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/posts\/3901","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/users\/6542"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/comments?post=3901"}],"version-history":[{"count":3,"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/posts\/3901\/revisions"}],"predecessor-version":[{"id":3931,"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/posts\/3901\/revisions\/3931"}],"wp:attachment":[{"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/media?parent=3901"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/categories?post=3901"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.uni-paderborn.de\/sse\/wp-json\/wp\/v2\/tags?post=3901"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}