{"id":140,"date":"2024-09-04T19:32:30","date_gmt":"2024-09-04T19:32:30","guid":{"rendered":"https:\/\/www-devel.ssc.wisc.edu\/?p=140"},"modified":"2026-10-01T14:41:18","modified_gmt":"2026-10-01T14:41:18","slug":"sscc-news-for-september-2024","status":"publish","type":"post","link":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/2024\/09\/04\/sscc-news-for-september-2024\/","title":{"rendered":"SSCC News for September 2024"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Welcome to the SSCC!<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We want to extend a warm welcome to all the new members of the Social Science Computing Cooperative, whether you\u2019re a new faculty member, staff member, or graduate student who will use our resources for research; or an undergraduate taking a class that uses SSCC resources.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What is the SSCC?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The SSCC provides servers, software, training, and consulting to support researchers (and future researchers) who do statistical analysis. If you didn\u2019t attend an orientation session, feel free to email the\u00a0<a href=\"https:\/\/www.sscc.wisc.edu\/get-help\/\">SSCC Help Desk<\/a>, tell us about yourself, and ask what we can do for you.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What is SSCC News?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">SSCC News is one of our main ways of getting information to our members. It comes out about once every two months. Please look over the email when you get it and then read the articles that will affect you.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you\u2019d rather not receive SSCC News, email\u00a0<a href=\"mailto:helpdesk@ssc.wisc.edu\">helpdesk@ssc.wisc.edu<\/a>\u00a0and they can take care of that for you. If you\u2019re no longer interested in SSCC News because you no longer use your SSCC account, they can close it for you.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Most Project Directories Will Move to ResearchDrive<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Last month, the SSCC Chairs &amp; Directors Committee voted to direct the SSCC not to buy new storage space for at least two years. Instead, as many project directories as possible will be moved to ResearchDrive, freeing up space for projects that cannot be moved. The SSCC will only create new project directories for data that cannot be stored in ResearchDrive. This will save SSCC members an estimated $150,000 over two years and more going forward, by taking advantage of the large investment the University has made in ResearchDrive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Note that projects in Silo, projects containing administrative data, and projects used for instruction cannot be moved to ResearchDrive. If your project is or soon will be larger than 25TB, the amount of space ResearchDrive provides free, we\u2019ll talk about what will best meet your needs. But you should plan on most other projects moving to ResearchDrive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Only PIs can request ResearchDrive space (see the\u00a0<a href=\"https:\/\/kb.wisc.edu\/researchdata\/129905\">eligibility rules<\/a>), but they can make it available to graduate students and others.\u00a0<strong>Graduate students who need project space should speak with their advisor or another faculty member and request space in that faculty member\u2019s ResearchDrive.<\/strong> Faculty, we are counting on you to make sure all grad students can store their data in ResearchDrive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you have a project directory, anticipate receiving an invitation to move your data to ResearchDrive in the near future. But you are welcome to start the process today: just read <a href=\"https:\/\/kb.wisc.edu\/sscc\/102687\">Using ResearchDrive<\/a> and fill out the <a href=\"https:\/\/uwmadison.co1.qualtrics.com\/jfe\/form\/SV_866xP26eJHvqoLj\">ResearchDrive Request Form<\/a>. As always, if you have questions please send them to the <a href=\"https:\/\/sscc.wisc.edu\/get-help\">SSCC Help Desk<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">SSCC Training<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The SSCC\u2019s statistical consultants will teach a variety of topical workshops yet this semester. Highlights include <a href=\"https:\/\/sscc.wisc.edu\/sscc_jsp\/training\/details.jsp?class_id=2936\">Using the SSCC Linux Servers<\/a>, <a href=\"https:\/\/sscc.wisc.edu\/sscc_jsp\/training\/details.jsp?class_id=2924\">Functions and Iteration in R<\/a>, <a href=\"https:\/\/sscc.wisc.edu\/sscc_jsp\/training\/details.jsp?class_id=2937\">Publication-Quality Tables in Stata<\/a>, and <a href=\"https:\/\/sscc.wisc.edu\/sscc_jsp\/training\/details.jsp?class_id=2923\">Missing Data Analysis with Blimp<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you missed our core R and Stata classes, we\u2019ll teach them again in January before the spring semester starts. They will be online so you don\u2019t have to be back in Madison yet. You can also find the material for those classes in our <a href=\"https:\/\/www.ssc.wisc.edu\/statistics\/\">Knowledge Base<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Coming Soon: A Database Server for Research<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In the coming weeks the SSCC will put a database server into production that is specifically designed for research and research data. It will run Postgres and MariaDB. This will allow researchers to work with large data sets that are best stored as databases and queried using SQL. If you\u2019re interested in using the new database server, reach out to the <a href=\"https:\/\/sscc.wisc.edu\/get-help\">SSCC Help Desk<\/a> and we\u2019ll keep you informed.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Reminder: Help Desk Changes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">While the SSCC Help Desk is short-handed, it will close earlier (3:00PM) and be available by email, voice mail, or appointment. See the <a href=\"https:\/\/sscc.wisc.edu\/get-help\">Help Desk web page<\/a> for more information.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Linux File Server Issues Resolved<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Over the summer, we had recurring issues with the Linux file server that affected everything done in Linux. It was one of the longest-running problems in the SSCC\u2019s history and we owe you an explanation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In late spring, we put a new and powerful Linux file server into production. The new server had a higher default for the maximum number of threads (parallel processes) the file server can use. Given that the Slurm cluster can have hundreds of jobs trying to write files at the same time, this seemed like a good thing. What we didn\u2019t know is that there is an obscure and apparently low-priority bug in Linux where file server threads can interfere with each other and cause the server to hang. Under \u201cnormal\u201d circumstances the probability of this occurring is low, but the probability increases non-linearly with the number of threads being used.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">At first we thought the problem was particular jobs that read and write lots of files: the problem would only occur when such jobs where running and went away when they were killed, but it was never consistent. (In reality the jobs only mattered in that they prompted the file server to use more threads.) Then we tried giving the server more resources: more cores, more memory, more network bandwidth\u2026and more threads (which only made things worse). We developed a variety of new tools for monitoring the servers and detecting problems. The whole process was slow because the problem only occurred every few days at most, so it took a long time to test theories or see if the latest change did any good. Meanwhile, the system administrators were monitoring the servers and clearing problems when they occurred on nights and weekends. Finding a solution to this problem was their <strong>top<\/strong> priority.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Finally, we found the bug report that explained what we were seeing and immediately reduced the maximum number of threads to what it had been on the old file server. That was now 27 days ago and the problem has not recurred, so we\u2019re reasonably confident it\u2019s fixed. The file server will sometimes be slow when it\u2019s being heavily used, but no longer hangs. We apologize for the trouble this issue caused, and you can rest assured no one was more frustrated by it than the system administrators involved.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Welcome to the SSCC! We want to extend a warm welcome to all the new members of the Social Science Computing Cooperative, whether you\u2019re a new faculty member, staff member, or graduate student who will use our resources for research; or an undergraduate taking a class that uses SSCC resources. What is the SSCC? The &hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_uw_seo_meta_title":"","_uw_seo_meta_description":"","_uw_seo_twitter_card_type":"summary_large_image","_uw_seo_meta_image":"","_uw_seo_meta_image_url":"","_uw_seo_meta_image_sizes":[],"_uw_seo_custom_meta_tags":[],"uw_external_link":"","footnotes":""},"categories":[4],"tags":[],"class_list":["post-140","post","type-post","status-publish","format-standard","hentry","category-sscc-news"],"_links":{"self":[{"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/posts\/140","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/comments?post=140"}],"version-history":[{"count":1,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/posts\/140\/revisions"}],"predecessor-version":[{"id":167,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/posts\/140\/revisions\/167"}],"wp:attachment":[{"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/media?parent=140"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/categories?post=140"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/tags?post=140"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}