{"id":142,"date":"2024-06-18T13:56:15","date_gmt":"2024-06-18T13:56:15","guid":{"rendered":"https:\/\/www-devel.ssc.wisc.edu\/?p=142"},"modified":"2026-10-01T14:41:18","modified_gmt":"2026-10-01T14:41:18","slug":"sscc-news-june-2024","status":"publish","type":"post","link":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/2024\/06\/18\/sscc-news-june-2024\/","title":{"rendered":"SSCC News June 2024"},"content":{"rendered":"\n

Welcome, New Members from the School of Education!<\/a><\/h2>\n\n\n\n

The School of Education (SoE) has joined the SSCC for a one-year pilot membership, after which we\u2019ll jointly evaluate a permanent membership. We\u2019ve scheduled several orientation sessions in the coming month, mostly (but not exclusively) for new members from SoE\u2013visit our training page<\/a> for details and to register. There will be more offered in the fall. Send any questions you have to the SSCC Help Desk<\/a>. We\u2019re eager to help you get started using SSCC resources.<\/p>\n\n\n\n

Changes to Storage Limits at the SSCC<\/a><\/h2>\n\n\n\n

Data storage has become a major expense for the SSCC. On average, it costs the SSCC $82 to store and back up 1TB of data for one year (compared to the $120\/TB\/year charge for ResearchDrive space beyond the 25TB given to every PI for free)<\/span> but in FY 2024 alone we\u2019ve spent roughly $130,000 on storage. We\u2019re asking all of our members who use significant amounts of storage to make sure they\u2019re using it as efficiently as possible. To encourage efficient use, we have implemented a new set of quotas for project directories and new tools to manage them.<\/p>\n\n\n\n

New project directories will start with a quota of 1TB while existing project directories have been given a quota equal to at least their current usage plus 20%. If you need more space you can request a quota of 5TB by visiting the SSCC self-service portal<\/a>. You must be on campus or connected to a campus VPN (UW\u2013Madison VPN, SSCC VPN, etc.) to access the portal. If you need more space beyond that, you can request a quota of 25TB using the same form. Your sponsoring agency will be informed of this request. If you still need more space, email the Help Desk<\/a> and we\u2019ll schedule a meeting to discuss your needs.<\/p>\n\n\n\n

Storing data as very large numbers of small files uses space inefficiently and slows down the file server. Putting many files in a single directory is especially problematic\u2013we suggest staying under about 1,000 files per directory. To maintain the performance of the file system, we have implemented a quota for project and home directories of either 1 million files or 20% more than the current file count, whichever is greater. You can request an increase to 5 million files for project directories using the portal. We will later reduce the maximum quota to five million files for all projects. This may require some changes to how you store your data; the SSCC\u2019s Help Desk staff and statistical consultants will be happy to help.<\/p>\n\n\n\n

We appreciate your help in making sure our shared computing resources are used as efficiently as possible.<\/p>\n\n\n\n

Default Linux Shell to Change to Bash<\/a><\/h2>\n\n\n\n

On July 19th<\/sup>, we will change the default Linux shell for most SSCC members from tcsh to bash. We will not change the shell of anyone who has customized tcsh. You can change your own shell by visiting the new self-service portal<\/a>.<\/p>\n\n\n\n

The \u201cshell\u201d is what you see when you first log into a Linux server. Many, many years ago the SSCC chose tcsh as our default shell because it made it easier to run long jobs (the kind of jobs that go to Slurm now). However, in the ensuing years, most of the world settled on bash because it\u2019s better for writing shell scripts. If you do a web search for how to do something on a Linux server, what you find will probably assume you\u2019re using bash. Most Linstat and LinSilo users don\u2019t need to worry about this change and won\u2019t notice any significant differences. But if you find you prefer tcsh, you can now change your shell at any time using the self-service portal<\/a>.<\/p>\n\n\n\n

SSCC Self-Service Portal<\/a><\/h2>\n\n\n\n

The SSCC self-service portal<\/a> is now available and can be used for things like setting your preferred email address, updating your affiliations, requesting more storage space, checking your current storage usage, and changing your Linux shell. You must be on campus or connected to a campus VPN (UW\u2013Madison VPN, SSCC VPN, etc.) to access the portal.<\/p>\n\n\n\n

Blimp Now Available On SSCC Servers<\/a><\/h2>\n\n\n\n

Blimp is a standalone program that uses Bayesian estimation to handle missing data. Blimp is both easier and more flexible than multiple imputation. We have documentation<\/a> to guide you on how to use Blimp on our servers.<\/p>\n\n\n\n

Someone Submitted 1,000 Jobs To Slurm! Now What?<\/a><\/h2>\n\n\n\n

A large research computing cluster like Slurm is designed to get as much work done as possible, not necessarily to get a particular job done as soon as possible. In that sense it\u2019s a good thing when new jobs can\u2019t start right away\u2013it means servers aren\u2019t sitting idle. Jobs having to wait a few hours before they start is not considered a problem on a big cluster, but even that\u2019s very rare at the SSCC. The Slurm Status<\/a> page will tell you how long jobs have had to wait over the past three days.<\/p>\n\n\n\n

Slurm prioritizes jobs based on how long they\u2019ve been waiting and how much compute time the user has used recently. So if someone submits 1,000 jobs to the cluster, they will very quickly become the very lowest priority. That doesn\u2019t mean their jobs will be preempted by other jobs (unless those jobs are submitted to a higher priority partition) but it does mean that when a job finishes and computing resources become available, Slurm will start up anyone else\u2019s job first. So just because you see 1,000 jobs in the Slurm queue does not mean you can\u2019t use Slurm\u2013get your jobs in the queue too and Slurm will start them in short order.<\/p>\n\n\n\n

On the other hand, that may be a good time to take advantage of partitions other than the default \u201csscc\u201d partition, like \u201cshort\u201d (includes servers reserved for jobs that will run for less than six hours) or \u201cecon-grad\u201d and \u201cecon-fac\u201d (servers where Economics grad students or faculty have priority because the Economics Department paid for part of the cost). The Slurm Partitions and Priorities<\/a> in the Guide to Research Computing at the SSCC<\/a> will tell you more.<\/p>\n\n\n\n

Using AI to Write Code<\/a><\/h2>\n\n\n\n

All UW\u2013Madison faculty, staff, and students now have access to Microsoft\u2019s Copilot<\/a> AI, including GPT-4. One of the more intriguing uses for large language models like Copilot or GPT is writing code, and the SSCC\u2019s statistical consultants have been experimenting with it. The results are in many ways impressive: it\u2019s quite good at understanding complex questions, especially if you include specific examples, and the code it generates is mostly right. The trouble is, code that is mostly right usually produces results that are completely wrong. Any code generated by AI needs to be checked carefully and often corrected.<\/p>\n\n\n\n

You cannot use Copilot as an alternative to learning how to code.<\/strong> Using code you don\u2019t understand will frequently lead to frustration, as you encounter problems you don\u2019t know how to solve. The biggest problem is that you can\u2019t be sure the code is actually doing what you want and giving you the correct results. (This is true whether the code came from AI or from the web.) Researchers who want to work with data should learn a language like R, Stata, or Python and how to work with data (\u201cdata wrangling\u201d) in that language.<\/p>\n\n\n\n

While you could ask Copilot to teach you to code, we humbly suggest SSCC\u2019s training<\/a> or Statistical Computing Knowledge Base<\/a> instead. Any learning materials written by a knowledgeable human will probably be better written and more trustworthy than what Copilot can produce. On the other hand, if you\u2019re working through online material on your own, you can ask Copilot questions and use it as a virtual instructor.<\/p>\n\n\n\n

Asking Copilot to write code you could easily write may or may not save you time, as understanding and checking the code it gives you may take just as long as writing it yourself.<\/p>\n\n\n\n

Copilot can really help you out if you know how to code in general but aren\u2019t sure how to carry out a particular task. If you explain the task in some detail, the code Copilot gives you will probably be either correct or easily fixable, and making the code work will probably be faster than figuring out the task yourself. And you will learn from Copilot\u2019s solution.<\/p>\n\n\n\n

We encourage you to take advantage of Copilot as you\u2019re writing code. Just be aware of its limitations and use it for what it\u2019s good at. In the end, you are responsible for the code that generates your results no matter how it was written.<\/p>\n\n\n\n

Computer Summer Cleaning<\/a><\/h2>\n\n\n\n

With Summer in full swing, now is a great time to take some time to do a quick review of your computer and see if any changes need to be made. Things like:<\/p>\n\n\n\n

    \n
  • Review the software installed on your machine to see if it is still needed.<\/li>\n
  • Go through your documents and other files to see if you need to reorganize or remove files you no longer need.<\/li>\n
  • If the SSCC does not manage your computer, make sure that it is updated with the latest version of your installed software and operating system to help keep it safe and secure.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"

    Welcome, New Members from the School of Education! The School of Education (SoE) has joined the SSCC for a one-year pilot membership, after which we\u2019ll jointly evaluate a permanent membership. We\u2019ve scheduled several orientation sessions in the coming month, mostly (but not exclusively) for new members from SoE\u2013visit our training page for details and to …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_uw_seo_meta_title":"","_uw_seo_meta_description":"","_uw_seo_twitter_card_type":"summary_large_image","_uw_seo_meta_image":"","_uw_seo_meta_image_url":"","_uw_seo_meta_image_sizes":[],"_uw_seo_custom_meta_tags":[],"uw_external_link":"","footnotes":""},"categories":[4],"tags":[],"class_list":["post-142","post","type-post","status-publish","format-standard","hentry","category-sscc-news"],"_links":{"self":[{"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/posts\/142","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/comments?post=142"}],"version-history":[{"count":1,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/posts\/142\/revisions"}],"predecessor-version":[{"id":169,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/posts\/142\/revisions\/169"}],"wp:attachment":[{"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/media?parent=142"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/categories?post=142"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www-devel.ssc.wisc.edu\/index.php\/wp-json\/wp\/v2\/tags?post=142"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}