README.md
... ...
@@ -2,5 +2,3 @@
2 2
![BioGrids](images/terminal.png)
3 3
4 4
This site provides information on how to install, use, and get help with your BioGrids software collection. We also include some information about BioGrids services. Please let us know if you have questions.
5
-
6
-THIS IS A TEMPORARY SITE.
... ...
\ No newline at end of file
_Sidebar.md
... ...
@@ -7,7 +7,6 @@ Support for Users
7 7
* [Overriding Software Versions](versions)
8 8
* [Install BioGrids: MacOS GUI](client_install)
9 9
* [Install BioGrids: CLI](client_CLI_install)
10
-* Just A Test - Again - three
11 10
12 11
Support for Site Administrators
13 12
* [Software Installation](installation_admin)
admin_overrides.md
... ...
@@ -0,0 +1,36 @@
1
+## Site Administrator Overrides
2
+
3
+### Software Blacklist
4
+Site administrators can prevent the shell initialization files from configuring software on a host by creating an empty file at
5
+
6
+ **/programs/local/conf/disabled-hosts/<hostname>**
7
+
8
+where **<hostname>** is the output of the //hostname -s// command. This is useful for encouraging users not to run jobs on bastion hosts, file servers, etc.
9
+
10
+### Administrator Software Version Overrides
11
+Most of the software titles installed in the SBGrid Suite have several versions available. Typically, we try to provide at least three versions of each software package, although very old or very new applications may have fewer versions.
12
+
13
+There are two methods for site admins to configure version overrides using files placed in a **/programs/local/conf/** directory.
14
+
15
+- Site-wide Version Overrides
16
+A file named **site.conf** will be read when any SBGrid shell is initialized from the installation.
17
+
18
+- Per-host Version Overrides
19
+A file named **<hostname.conf>**, where **<hostname>** is the output of the //hostname -s// command, will be read after the site.conf on the host matching that hostname.
20
+
21
+### Precedence
22
+There are three files that will be examined for version overrides during shell initialization. In order from lowest to highest precedence:
23
+
24
+ * /programs/local/conf/site.conf
25
+ * /programs/local/conf/<hostname.conf>
26
+ * $HOME/.sbgrid.conf
27
+
28
+The overrides are reported during shell initialization:
29
+
30
+```
31
+ Configuration options in effect for this shell:
32
+ Variable Setting Configuration
33
+ SPIDER_L 18.10 site
34
+ CNS_L 1.21 machine
35
+ CCP4_L 6.1.2 personal
36
+```
client_CLI_install.md
... ...
@@ -0,0 +1,129 @@
1
+## SBGrid installation client CLI
2
+
3
+This page is for the SBGrid CLI (command line interface) software installation client for computers running Linux (CentOS/RHEL 6 or 7 recommended) or Apple OSX computers running MacOSX v10.9 - 10.12.
4
+
5
+Looking to get started with the SBGrid installation client? First, set up an account by registering here : [SBGrid registration](https://sbgrid.org/registration/register/)
6
+
7
+A GUI version is also available for MacOS / OSX - [installation instructions can be found here](client_install).
8
+[GUI usage info is here](client_usage).
9
+
10
+### Pre-installation requirements
11
+
12
+**Admin access required on your computer.**
13
+ The SBGrid installation client will install the software tree to /opt/sbgrid and create a symlink from this directory to /programs. While admin privileges are not required to run the application, you will need admin privileges to create /opt/sbgrid and /programs .
14
+
15
+**Existing SBGrid installations must be renamed or removed.**
16
+ If you have previously installed SBGrid software on your computer at /programs or at /opt/sbgrid, these directories should be removed or renamed. The client will create these on first run.
17
+
18
+**Hard Drive Space**
19
+ Depending on what you install, ensure you have enough free hard drive space on your machine.
20
+
21
+**For large packages (Phenix, Rosetta, CCP4), a wired connection is recommended.**
22
+ Some packages are quite large and can be slow to download, especially over WiFi.
23
+
24
+**pip**
25
+ pip is a package management system used to install and manage software packages written in Python. The software installation client is installable by pip. This may not be on you machine by default, instructions for adding it are included here.
26
+
27
+### pip - Mac Installation
28
+
29
+ On mac, pip is available from [macports](https://www.macports.org/), [homebrew](http://brew.sh/), and [fink](http://finkproject.org/). It also can be added using **easy_install**. easy_install will install pip into your system python.
30
+
31
+ $ sudo easy_install pip
32
+
33
+###pip - Linux Installation
34
+
35
+ On CentOS6, you can install pip from the EPEL repository (yum -y install python-pip) or as [per these directions](https://pip.pypa.io/en/stable/installing/)
36
+
37
+## CLI client - Installation and activation
38
+**1. Download the latest SBGrid installation client from here.**
39
+
40
+[Latest build : 1.0.471, 2017-02-17](downloads/SBGrid_Installer-1.0.471-py2-none-any.whl)
41
+
42
+**2. Install / Upgrade with pip**
43
+
44
+ $ sudo pip install SBGrid_Installer-1.0.461-py2-none-any.whl
45
+
46
+ Which should produce output similar to :
47
+
48
+```
49
+Unpacking ./SBGrid_Installer-1.0.448-py2-none-any.whl
50
+Requirement already satisfied (use --upgrade to upgrade): pyasn1 in /nfs/programs-dev/i386-mac/python/2.7.5/lib/python2.7/site-packages (from SBGrid-Installer==1.0.488)
51
+Downloading/unpacking requests (from SBGrid-Installer==1.0b2.dev412)
52
+ Downloading requests-2.11.1.tar.gz (485kB): 485kB downloaded
53
+ Running setup.py egg_info for package requests
54
+
55
+ warning: no files found matching 'test_requests.py'
56
+Installing collected packages: SBGrid-Installer, requests
57
+ Running setup.py install for requests
58
+
59
+ warning: no files found matching 'test_requests.py'
60
+Successfully installed SBGrid-Installer requests
61
+Cleaning up...
62
+```
63
+
64
+The *sbgrid* command will produce this usage info :
65
+
66
+ $ sbgrid
67
+
68
+```
69
+usage:
70
+sbgrid <command> [<command_args>]
71
+
72
+Available commands are:
73
+ activate Activate new installation
74
+ list List available packages
75
+ info Print package metadata
76
+ install Install single package
77
+ install-all Install all packages from collection
78
+ installed List installed packages
79
+ remove Remove installed package
80
+ resume Resume interrupted installations
81
+ upgrades List available upgrades for installed packages
82
+ upgrade Upgrade package to latest version
83
+ upgrade-all Upgrade all packages to latest version
84
+ version Print package version string
85
+sbgrid: error: too few arguments
86
+
87
+```
88
+
89
+**4. Activate the installation**
90
+Enter your site and key which you should have received these by email.
91
+
92
+ $ sbgrid activate <site name> <user name> <activation key>
93
+
94
+You may be prompted for your sudo password to create /programs and /opt/sbgrid.
95
+
96
+**5. On successful activation, list software titles.**
97
+
98
+ $ sbgrid list
99
+
100
+ If activation continues to fail, you may have an old SBGrid installation at /programs or /opt/sbgrid. These should be removed.
101
+
102
+You may also be blocked from accessing SBGrid servers on port 873 and port 8080 by your institution's firewall. You can check if you can access [port 873 here](http://portquiz.net:873) and [port 8080 here](http://portquiz.net:8080)
103
+
104
+For help, email *bugs@sbgrid.org.*
105
+
106
+### Installation on an external drive or USB key
107
+Currently the installer application installs the software tree in /opt/sbgrid and uses a symlink at /programs. This link is required for the software to work, but in principle the software can be located anywhere and linked to /opt/sbgrid. This functionality is not yet included in the SBGrid application, but can be set up manually with a symlink.
108
+
109
+An installation can be made on an external drive if you first do an install (into /opt/sbgrid), then move it and symlink to the new location.
110
+
111
+**Example:** 128Gb USB 'key' drives are available on Amazon for 30$ or less and are a nice way to supplement the SSDs that come in Mac laptops. To install on an external drive like this, mount it on your computer. For this example, the mount is */Volumes/sbgrid_client_install*. Then, using the SBGrid installer, do an initial installation on the hard drive. To make life easy, just do one application. Then close the installer application.
112
+
113
+Next move */opt/sbgrid* to the USB key at */Volumes/sbgrid_client_install*. You will likely need sudo for this.
114
+
115
+ $ sudo mv /opt/sbgrid /Volumes/sbgrid_client_install/.
116
+
117
+Now we have */Volumes/sbgrid_client_install/sbgrid*. Create a symlink from */opt/sbgrid* to */Volumes/sbgrid_client_install/sbgrid*.
118
+
119
+ $ sudo ln -s /Volumes/sbgrid_client_install/sbgrid /opt/sbgrid
120
+
121
+The symlink at */programs* still points to */opt/sbgrid*, and that is a link that points to the external location. Open the installer and add the desired applications. These will now install to the external drive.
122
+
123
+[SBGrid install client -- Usage info](client_usage)
124
+
125
+### Known issues
126
+
127
+* **rsync required on 873 or 8080**. The application will fail if outbound port 873 and port 8080 are blocked by your institution or are otherwise not available.
128
+
129
+* **Activation Failed**. In some cases the Authorization dialog for admin privileges does not appear during activation. This causes the client activation to fail. This is usually due to a previously existing /programs directory or symlink from a prior installation. This must be removed for activation to be successful. Other users have reported successful activation after a restart.
client_install.md
... ...
@@ -0,0 +1,67 @@
1
+## SBGrid installation client
2
+
3
+This page is for the SBGrid GUI installation client for Apple OSX computers running MacOSX v10.9 - 10.12 .
4
+
5
+![](images/client_all.png)
6
+
7
+Looking to get started with the SBGrid installation client? First, set up an account by registering here : [SBGrid registration](https://sbgrid.org/registration/register/)
8
+
9
+A command line version is also available for mac and linux - installation instuctions can be found here : [CLI installation](client_CLI_install).
10
+
11
+Usage info for the GUI and command line clients can be found here : [GUI Usage](client_usage) and here : [CLI usage](client_CLI_usage) respectively.
12
+
13
+### Pre-installation
14
+
15
+**1. Admin access required.**
16
+ The SBGrid installation client will install the software tree to */opt/sbgrid* and create a symlink from this directory to */programs*. While admin privileges are not required to run the application, you will need admin privileges to create */opt/sbgrid* and */programs* .
17
+
18
+**2. Existing installations must be renamed or removed.**
19
+ If you have previously installed SBGrid software on your computer at */programs* or at */opt/sbgrid*, these directories should be removed or renamed. The client will create these on first run.
20
+
21
+**3. Hard Drive Space**
22
+ Depending on what you install, ensure that you have enough free hard drive space on your machine.
23
+
24
+**4. For large packages (Phenix, Rosetta, CCP4), a wired connection is recommended.**
25
+ Some packages are quite large and can be slow to download, especially over WiFi.
26
+
27
+### Installation and activation
28
+
29
+**1. Download the latest SBGrid installation client from here.**
30
+
31
+ [Latest build : 1.0.471, 2017-02-17](downloads/SBGrid_Installer-1.0.471.dmg)
32
+
33
+**2. Mount the .dmg disk image on your Mac.** Copy the 'SBGrid installer.app' to a convenient location - it does not need to be in /Applications. The .app bundle can be run directly from the mounted dmg image with no installation if desired.
34
+
35
+**3. Enter your site, user name, and key.** You should have received these by email.
36
+
37
+![activation](images/client_activation.png)
38
+
39
+On activation you will be prompted to provide admin credentials for your mac. This is to create the */opt/sbgrid* directory and */programs* symlink.
40
+
41
+![activation](images/client_permissions.png)
42
+
43
+**4. On successful activation, you should see the client populate with software titles.** If not, please close the application and try again. If activation continues to fail, you may have an old SBGrid installation at */programs* or */opt/sbgrid*. These should be removed. You may also be blocked from accessing SBGrid servers on port 873 and port 8080 by your institution's firewall. You can check if you can access [port 873 here](http://portquiz.net:873) and [port 8080 here](http://portquiz.net:8080)
44
+For help, email *bugs@sbgrid.org.*
45
+
46
+### Installation on an external drive or USB key
47
+Currently the installer application installs the software tree in */opt/sbgrid* and uses a symlink at */programs*. The */programs* link is required for the software to work, but in principle the software can be anywhere by linking */opt/sbgrid* to the desired installation point. This functionality is not yet included in the SBGrid application, but can be set up manually post-install with a symlink. First do an install (into */opt/sbgrid*), then move it to the desired localtion. Then symlink to the new location from */opt/sbgrid*.
48
+
49
+**Example:** 128Gb USB 'key' drives are available on Amazon for 30$ or less and are a nice way to supplement the small SSDs that come in Mac laptops. To install on an external drive in this way, first mount it on your computer. For this example, the mount is */Volumes/sbgrid_client_install*. Then, using the SBGrid installer, do an initial installation on the hard drive. To make life easy, just do one application. Then close the installer application.
50
+
51
+Next move **/opt/sbgrid** to the USB key at */Volumes/sbgrid_client_install*. You will likely need sudo for this.
52
+
53
+ sudo mv */opt/sbgrid* /Volumes/sbgrid_client_install/.
54
+
55
+Now we have */Volumes/sbgrid_client_install/sbgrid*. Create a symlink from **/opt/sbgrid** to */Volumes/sbgrid_client_install/sbgrid*.
56
+
57
+ sudo ln -s /Volumes/sbgrid_client_install/sbgrid */opt/sbgrid*
58
+
59
+The symlink at **/programs** still points to **/opt/sbgrid**, and that is a link that points to the external location. Open the installer and add the desired applications. These will now install to the external drive.
60
+
61
+[SBGrid install client -- Usage info](client_usage)
62
+
63
+### Known issues
64
+
65
+* **rsync required on 873 or 8080**. The application will fail if outbound port 873 and port 8080 are blocked by your institution or are otherwise not available.
66
+
67
+* **Activation Failed**. In some cases the Authorization dialog for admin privileges does not appear during activation. This causes the client activation to fail. This is usually due to a previously existing */programs* directory or symlink from a prior installation. This must be removed for activation to be successful. Other users have reported successful activation after a restart.
developers.md
... ...
@@ -0,0 +1,26 @@
1
+## SBGrid Developer Support Program
2
+In an effort to show our appreciation for their hard work and to encourage the continuing development and support of cutting edge structural biology software, SBGrid offers a Developer Support Program featuring a wide array of benefits for structural biology software developers.
3
+
4
+### Developer Network
5
+We can provide access to a dedicated development and test network for software developers. As a developer of an SBGrid supported application, you can request an account on this network and use our infrastructure to build and test your application on a wide range of operating systems. Visit the [developer network page](resources) for more details.
6
+
7
+### Support Letters
8
+When applying for grants or financial support please let us know! We can provide support letters on behalf of members of SBGrid Consortium. Letters would include details about our membership, number of computers that can access your software, operating systems used by our community, etc. Please use the [contact form](http://sbgrid.org/contact.php|contact) on the SBGrid website.
9
+
10
+### Software Beta Testing
11
+You are welcome to use SBGrid laboratories to beta-test your software. Just let us know when the new beta is available, and we will install it for all participating laboratories or a limited subset that you choose.
12
+
13
+### SBGrid Consortium Membership
14
+If your application is under active maintenance or development, your development group is entitled to free membership in the consortium. Letting us maintain your general structural biology software environment leaves you with more time to concentrate on your unique software contribution to the community.
15
+
16
+### Annual Software Awards
17
+Members of SBGrid Consortium vote in an annual poll to select their favorite structural biology application. Paul Emsley was the winner of the first poll. He won a Macbook Air laptop for his application Coot.
18
+
19
+### Stretch Your Resources
20
+First level software support is provided by the SBGrid non-profit center at Harvard Medical School. All members of the consortium communicate with the service center for assistance with software installation, upgrades, and basic troubleshooting. For many of the mundane problems involving installation, operating system portability and environmental set up, SBGrid members will never need to contact you for support. Your expertise with the software be best utilized answering questions and working on problems that directly involve the science behind your application.
21
+
22
+### Broad Exposure for Your Software
23
+More than 140 structural biology laboratories from 50 institutions in 11 countries participate in the Consortium including the labs of 25 HHMI investigators and 2 Nobel Prize winners. Deployment of your application can be customized to include only non-profit labs or broadened to include industry labs as well. Please visit the Consortium website for the full list of participating groups that can benefit from rapid access to your software. Please note that most SBGrid Consortium laboratories standardize on hardware and operating systems, which simplifies the installation and support process.
24
+
25
+### Connect With Users
26
+Announcing a new version of your software? We can use webex to broadcast your presentation to all laboratories participating in our consortium. We can also disseminate information about your application to our community in our monthly newsletter. Visiting Boston? Please let us know because we'd love to schedule a seminar for members of our local structural biology community.
getting_started.md
... ...
@@ -0,0 +1,47 @@
1
+## Getting Started
2
+
3
+### Using the SBGrid software collection
4
+The SBGrid software collection can be found installed at **/programs**.
5
+All software in the SBGrid collection is run from the the command line in a terminal. Do set up the shell environment the terminal to run the programs, you will need to initialize the SBGrid environment.
6
+
7
+Bash users should initialize the SBGrid environment using :
8
+
9
+ source /programs/sbgrid.shrc
10
+
11
+Tcsh users should initialize the SBGrid environment with :
12
+
13
+ source /programs/sbgrid.cshrc
14
+
15
+On initialization, you should see the SBGrid welcome message
16
+
17
+![SBGrid](images/terminal1.png)
18
+
19
+### Making the shell configuration automatic
20
+
21
+To setup your terminal to automatically load the SBGrid environment, add the following to your configuration file:
22
+
23
+For Bash, in **~/.bashrc**, add :
24
+
25
+ if [ -f /programs/sbgrid.shrc ] ; then
26
+ source /programs/sbgrid.cshrc
27
+ fi
28
+
29
+For tcsh, in **~/.tcshrc** or **~/.cshrc**, add :
30
+
31
+ if ( -e /programs/sbgrid.cshrc) then
32
+ source /programs/sbgrid.cshrc
33
+ endif
34
+
35
+### Setting an alias
36
+
37
+If you would prefer to load the SBGrid environment on demand and not have to type the command above, you can set an alias to a shorter command of your choosing.
38
+
39
+For example, for bash in ~/.bashrc :
40
+
41
+ alias sbg='source /programs/sbgrid.shrc'
42
+
43
+For tcsh in ~/.tcshrc, this would be
44
+
45
+ alias sbg 'source /programs/sbgrid.cshrc'
46
+
47
+Questions or problems? Please email **bugs@sbgrid.org**
gpu.md
... ...
@@ -0,0 +1,21 @@
1
+## GPU computing
2
+
3
+GPU-accelerated computing uses a graphics processing unit (GPU) to accelerate computations in computer programs. Several applications in the SBGrid programs tree are GPU-accelerated. Some examples are RELION, GCTF, MotionCor2, Gautomatch, IMOD, etc. Here's what you need to take advantage of a GPU:
4
+
5
+### Requirements for GPU (CUDA) applications
6
+
7
+**NVIDIA Hardware**
8
+
9
+To use GPU-accelerated applications, you will need an NVIDIA GPU. We don't currently support other co-processors (Xeon Phi, AMD). If you think we should, let us know at bugs@sbgrid.org .
10
+
11
+ Not all GPUs are created equal - some applications are restricted to GPUs with a certain 'Compute Capability (i.e. of 3.5 or greater)'. You can check your card's rating and find more information on GPU computing here : [https://developer.nvidia.com/cuda-gpus](https://developer.nvidia.com/cuda-gpus)
12
+
13
+**Drivers**
14
+
15
+You linux workstation must have the proper drivers for your GPU. You can find drivers for your card here : [https://developer.nvidia.com/cuda-zone](https://developer.nvidia.com/cuda-zone)
16
+
17
+We currently support applications that rely on versions of CUDA 5 to 8 and these libraries are included with the SBGrid distribution. For best results, it is important to have a recent driver version if possible.
18
+
19
+For more information on GPUs and RELION, check Erik Lindahl's blog at the Department of Biochemistry & Biophysics, Stockholm University, [http://www.cryoem.se/relion-gpu](http://www.cryoem.se/relion-gpu)
20
+
21
+GPU questions? Want to use GPU support in an application and it's not working? Please email *bugs@sbgrid.org*
help.md
... ...
@@ -0,0 +1,15 @@
1
+## Software Help
2
+
3
+If you are having a problem with software included in the SBGrid suite or think you've discovered an application bug, please use the [SBGrid](http://sbgrid.org/bugs) web form to alert us to the problem. You can also email bugs@sbgrid.org with details on the problem, but the web form is the preferred method.
4
+
5
+If the problem with the software started after an update, you can configure your SBGrid environment to [use an older version of the software](overrides).
6
+
7
+### New Software and Software Updates
8
+To request a new software package or an update to an existing package, please use the [Software](http://sbgrid.org/req) Update/Installation Request form]].
9
+
10
+We try to accommodate most requests for new software, but we do occasionally run into difficulties. Despite most of our software being free for academics, some developers insist that each lab download and install their own software. We are happy to provide developers with the list of SBGrid members that have access to their software, but that doesn't always satisfy their requirements.
11
+
12
+Even when things go smoothly, license requests and working out licensing details with developers can take some time, so please don't wait until the last minute before requesting software you anticipate you may need for your project.
13
+
14
+### Newsletter
15
+SBGrid sends a monthly newsletter with information concerning software updates, upcoming webinars, talks and other items of interest to our community. The archive of previous newsletters is available online at the [SBGrid Newsletter Archive](https://sbgrid.org/newsletters/).
installation_admin.md
... ...
@@ -0,0 +1,51 @@
1
+##Managing Your Software Installation
2
+
3
+### *sbgrid-admin* Utility
4
+The *sbgrid-admin* utility used to install the software has a few other uses. There is a download link in the initial site set up email, and it is included in the installation itself at */programs/share/sbgrid/bin/sbgrid-admin*. It includes built-in help:
5
+
6
+```
7
+ $ sbgrid-admin -h
8
+sbgrid-admin: a tool for administering an SBGrid Software suite installation
9
+
10
+Usage: sbgrid-admin [-chirstu]
11
+
12
+ -c Cron mode. Used for automated updates of the software suite.
13
+ -h Help message.
14
+ -i Install the software suite on this machine.
15
+ -p Print a launchd plist for updating the software on OS X machines.
16
+ -r Regenerate sh/csh shell configuration files.
17
+ -s Install the software branch for SGI IRIX.
18
+ -t Test the configuration file.
19
+ -u Force manual update of an existing installation.
20
+```
21
+
22
+The most commonly used option is *sbgrid-admin -u* to force a manual update. This will synchronize your local installation with the SBGrid master installation hosted at Harvard Medical School.
23
+
24
+The SBGrid software configuration files use both static and dynamically generated files. The main shell initialization files, **/programs/sbgrid.cshrc** and **/programs/sbgrid.shrc** are static files, but the operating-system specific configurations are dynamically generated and customized for the packages available at your site. *sbgrid-admin -r* will regenerate the branch-specific configuration files should you accidentally delete or alter them.
25
+
26
+ Configuration Options
27
+The *sbgrid-admin* script has some additional configuration options to accommodate the installation requirements at different sites.
28
+
29
+The ~sbgrid/.sbgridrc configuration file for the *sbgrid-admin* script must contain a sitename and site key. All other configurations are optional.
30
+
31
+Options take the form of *OPTION_NAME="setting"*.
32
+
33
+| **Option** |** Example** | **Description** |
34
+|:------------- |:-------------:| :---------------|
35
+| SB_SITENAME | sbgrid-bene | The site name for your installation provided by SBGrid |
36
+| SB_SITEKEY | xyxyxyxyxyxyxy | The installation key for your site provided by SBGrid |
37
+| INSTALL_TARGET| /nfs/programs | The installation directory for the software |
38
+| RSYNC_ARGS | "--port 8080"| Advanced arguments to rsync. Can be used to workaround blocked rsync ports. |
39
+| HTTP_PROXY | host.name:port | Proxy hostname and port for use by rsync and curl |
40
+| POST_UPDATE_SCRIPT | /absolute/path/to/script|Optional script to be run after a successful software update |
41
+| CRON_SCHEDULE | 5 1 * * * | Set a custom cron schedule |
42
+
43
+
44
+ Rsync Transaction Logging
45
+If you want a date-stamped log of all rsync downloads, deletions and updates, you can add a line like this to the ~sbgrid/.sbgridrc file:
46
+
47
+```
48
+RSYNC_ARGS="--log-file=$HOME/logs/sbgrid-admin-`date +%Y%m%d`.log"
49
+```
50
+
51
+This will create a date-stamped log file any time an actual change to the software tree happens. In general this should not be necessary, but for troubleshooting purposes it can be useful.
mailinglist.md
... ...
@@ -0,0 +1,5 @@
1
+## Mailing List for Site Administrators
2
+
3
+We have a small mailing list for technical users and site administrators.
4
+
5
+Visit the [SBGrid-tech mailing list](https///cmcd.hms.harvard.edu/sympa/info/sbgrid-tech) page to sign up.
managing_install.md
... ...
@@ -0,0 +1,61 @@
1
+# HOWTOs: SBGrid Installation Notes
2
+
3
+## Site Configuration
4
+
5
+The SBGrid Software Suite is designed to be installed on a single computer at your site, which will then share that installation with all the workstations on your network via NFS.
6
+We refer to this computer as the site "master" server.
7
+
8
+The master server will require at least 250 GB of free disk space on a case-sensitive file system and must run some variety of UNIX. The installation script will download the software branches you select, and then install a cron job that will run regularly to check for updates.
9
+
10
+Software installation and update are initiated from your local server at your site and downloaded from our servers. http is used to query our servers to check for update, and rsync is used to download the updates. Outbound HTTP, and RSYNC is necessary.
11
+
12
+![NFS-based installation](images/site_diagram.png "NFS-based installation")
13
+
14
+## Pre-Installation
15
+
16
+| Installation Requirements | |
17
+| ------------------------- | ----------------------------------------------------|
18
+| Hard Drive Space | ~400 GB required |
19
+| Operating System | Linux, OS X and other UNIX |
20
+| Privileges | Root account (Linux) or Administrator account (Mac) |
21
+| network | Outbound http, rsync (ports 80 and 873, 8080) |
22
+
23
+
24
+### Approximate size per Branch
25
+| Branch | size |
26
+| ------------ | ------ |
27
+| i386-mac | 175Gb |
28
+| x86_64-linux | 210Gb |
29
+| shared files | 18Gb |
30
+
31
+[Size breakdown of titles in a typical installation](size_graph)
32
+
33
+Email [accounts@sbgrid.org](accounts@sbgrid.org) to request the installation script for the software. Be sure to include the following information in your email:
34
+
35
+ * Lab Name
36
+ * Types of computers on site: Linux, OS X Intel, OS X PowerPC, IRIX
37
+ * The names and email addresses of lab members that will use the software. One user may be designated to receive the majority of our correspondence if you would prefer not to hear from us too often.
38
+
39
+Once the email has been received and your information has been entered into our system, we will email you a script you will run that will begin the remote installation process. Instructions for running the script will be included in that email, and are reproduced below for reference.
40
+
41
+## Installation
42
+
43
+ 1. Create an 'sbgrid' user on the machine that will host your software installation. This should be a normal, unprivileged user account.
44
+ 2. Log in as this 'sbgrid' user.
45
+ 3. Download the installation script from the URL provided in the account creation email.
46
+ 4. Make it executable: `chmod +x sbgrid-admin`
47
+ 5. Run the script: `./sbgrid-admin -i`
48
+ 6. The script will walk you through configuring your installation, downloading the software and installing a cron job that will keep the software updated at your site. You will need the **site name** and **install key** provided in the account creation email. The installation may take anywhere from several hours to a day, depending on your network and the number of applications available to your site.
49
+
50
+## Post-Installation Steps
51
+
52
+ 1. Add a symlink in the root directory of each machine that wants to use the software. The installation script will print the suggested syntax for the command.`ln -s /path/to/sbgrid/installation /programs`
53
+ 2. Any user that wishes to use the software needs to configure their shell to initialize the SBGrid shell environment. For bash and other sh-compliant shells, add the following line to your ~/.bash_profile or ~/.bashrc: `. /programs/sbgrid.shrc` For csh and tcsh, add the following line to your ~/.cshrc: `source /programs/sbgrid.cshrc`
54
+ 3. For some linux distributions, the shipped version of tcsh does not support shell variables longer than 4096 characters. Since the current PATH in our default shell initialization is larger than that, you will need to update tcsh. RPMs for Red Hat 4/5 and CentOS 4/5 are provided at /programs/i386-linux/system/RPMS.
55
+
56
+
57
+## SGI IRIX Software Branch
58
+
59
+Since the IRIX software branch is frozen and no longer receiving updates, it is not installed and updated in the same manner as the active software branches. In order to install the IRIX software branch, please run: `./sbgrid-admin -s`
60
+
61
+Note that the IRIX branch is only compatible with *tcsh*; there is no sh-compatible configuration for this branch.
operatingsystems.md
... ...
@@ -0,0 +1,35 @@
1
+## Supported Operating Systems
2
+
3
+The SBGrid Software Suite can run on Linux, OS X and IRIX machines.
4
+
5
+### Linux
6
+We build and test software under 64-bit Red Hat 6. The majority of our Linux users are using the Red Hat/CentOS/Scientific Linux or Fedora distributions, but we also have labs using the software with Ubuntu, Debian and OpenSuSE. We don't explicitly test the software on these distributions, but it is known to work, and we will work to resolve problems reported by users of these Linux distributions.
7
+
8
+We officially support the two most recent releases of Red Hat/CentOS/Scientific Linux (currently 6 and 7) and the two most recent releases of Fedora Linux. At this time, only 64-bit versions of linux are fully supported. As of January 2015, we are no longer updating the 32-bit software branch, though we will work to resolve reported problems.
9
+
10
+Currently Supported:
11
+
12
+* Red Hat Enterprise Linux 6.x and 7.x and the community supported versions: CentOS and Scientific Linux.
13
+* We are no longer building software for RHEL 5.x, but compatible applications will remain in the software tree. We will respond to support requests for these and we will resolve problems where possible.
14
+
15
+## OS X Intel
16
+We build and test the programs under the most five most recent OS X releases.
17
+
18
+Currently Supported:
19
+
20
+* OS X Intel 10.9 - 10.12
21
+* As above with earlier versions of linux, there are many software applications that are fully functional on earlier versions of MacOSX in the SBGrid tree. We work to maintain an environment compatible with these OS versions, though they may not receive the latest applications available.
22
+
23
+## OS X PowerPC
24
+The OS X PowerPC branch is currently frozen. The applications are stable and work well on our PowerPC hardware, but they are not being updated.
25
+
26
+Currently Supported:
27
+
28
+* OS X PowerPC 10.4 - 10.5
29
+
30
+## IRIX
31
+The IRIX programs branch is currently frozen. The applications are stable and work well on our SGI hardware, but they are not being updated.
32
+
33
+Currently Supported:
34
+
35
+* IRIX 6.5.26 + bugfix releases
recommended.md
... ...
@@ -0,0 +1,7 @@
1
+# Recommended Hardware
2
+
3
+## Linux Workstations
4
+SBGrid's preferred hardware vendor is [ThinkMate](http://thinkmate.com) and we have some [recommended configurations](http://www.thinkmate.com/sbgrid) for structural biology computing. Please feel free to contact us at help@sbgrid.org regarding upcoming hardware purchases if you have questions - we'd be happy to advise.
5
+
6
+## Apple Workstations
7
+Any Apple machine can be a capable computer for structural biology. The basic Apple educational discount is available through their online web store, and your institution may have negotiated an even better deal with their Apple representative. SBGrid has several labs that run exclusively on Macs and OS X.
resources.md
... ...
@@ -0,0 +1,85 @@
1
+# Development Network
2
+
3
+The basic account is a UNIX shell account accessible via SSH. This network account will let you access designated machines in the developer network. Along with the shell account you also get up to 10 GB of data storage, access to high performance compilers, basic development tools like version control systems (svn, git, etc) and shell text editors.
4
+
5
+Once you have an account, you can access the developer network by connecting via SSH to:
6
+
7
+ *developer.sbgrid.org*
8
+
9
+ developer runs Scientific Linux 6 x86_64
10
+
11
+This machine (internally named sbgrid-dev-architect) is the only external facing host on the network, and it hosts your home directory and acts as the SSH bastion host for the network. From this machine you can SSH to any of the build and test hosts via their hostname in the table below in order to compile software, run tests, etc.
12
+
13
+The following are the always-on operating system/hardware combinations. They are generally kept at the latest point release:
14
+
15
+| Hostname | Operating System | Architecture | Notes |
16
+| ------------------ | --------------- | ----------- | ----------------------------------------- |
17
+| sbgrid-c5b | CentOS 5 | x86 | base SBGrid 32-bit Linux build VM |
18
+| sbgrid-c5b-64 | CentOS 5 | x86_64 | base SBGrid 64-bit Linux build VM |
19
+| sbgrid-c5t-64 | CentOS 5 | x86_64 | minimal install for portability testing |
20
+| sbgrid-c6b | CentOS 6 | x86 | base SBGrid 32-bit Linux build VM |
21
+| sbgrid-c6b-64 | CentOS 6 | x86_64 | base SBGrid 64-bit Linux build VM |
22
+| sbgrid-c6t-64 | CentOS 6 | x86_64 | minimal install for portability testing |
23
+| sbgrid-dev-vm-07 | Debian 5 | x86 | |
24
+| sbgrid-dev-vm-08 | Debian 5 | x86_64 | |
25
+| sbgrid-dev-vm-16 | Debian 6 | x86 | |
26
+| sbgrid-dev-vm-17 | Debian 6 | x86_64 | |
27
+| sbgrid-dev-vm-03 | Ubuntu 10.10 | x86 | |
28
+| sbgrid-dev-vm-04 | Ubuntu 10.10 | x86_64 | |
29
+| sbgrid-dev-vm-14 | OpenSuSE 11.3 | x86 | |
30
+| sbgrid-dev-vm-15 | OpenSuSE 11.3 | x86_64 | |
31
+| sbgrid-m106 | OS X 10.6 | x86/x86_64 | Base SBGrid OS X Intel build machine |
32
+| sbgrid-m107 | OS X 10.7 | x86/x86_64 | |
33
+| sbgrid-m108 | OS X 10.8 | x86/x86_64 | |
34
+| sbgrid-m109 | OS X 10.9 | x86/x86_64 | |
35
+| sbgrid-m1010 | OS X 10.10 | x86/x86_64 | |
36
+| sbgrid-dev-flange | OS X 10.5 | PowerPC | |
37
+
38
+
39
+We have the Intel, Portland and Absoft (PPC) compilers:
40
+
41
+* Intel 12.0, 11.1, 10.1 (ifc, ifort) - any linux host, OS X Intel hosts
42
+* Portland 14.4, 12.6, 10.9, 10.2, 9.0-1 (pgcc, pgf77/90/95) - sbgrid-dev-architect, sbgrid-dev-moose
43
+
44
+There is a configuration file for sh-compatible shells that will enable the compilers and their environmental settings:
45
+
46
+ $ . /build/conf/buildenv.sh
47
+
48
+Then you can use the 'build' function to control your compiler settings:
49
+
50
+ $ build help
51
+ build: a shell function for controlling compilers
52
+
53
+ Usage: build (help3264list)
54
+
55
+ help This help message
56
+ 32 Configure 32-bit compilers
57
+ 64 Configure 64-bit compilers
58
+ list List available compilers
59
+
60
+Additionally, there are other operating systems available as virtual machines that can be enabled (OpenSuSE, Ubuntu, Fedora, etc), and some additional hardware that can be made available (Sun SPARC, IRIX MIPS, etc) on request.
61
+
62
+Graphical access to the machines may be available through Apple Remote Desktop, VNC or the NoMachine desktop sharing client. Other tools could be made available if necessary as well. The whole thing's a bit of an experiment, really, so email us at `<bugs@sbgrid.org>`, and we'll see what we can work out.
63
+
64
+# Request an Account
65
+
66
+Thanks for your interest in participating in the SBGrid Developer Network.
67
+
68
+In order to be eligible for an account, you must have software in the SBGrid software suite or be planning to make your software available through SBGrid. This requirement limits accounts to developers of scientific software used in structural biology and related disciplines.
69
+
70
+That's the only requirement for new accounts. In return for access to these resources, you must agree to not abuse the resources for:
71
+
72
+* sending spam
73
+* cracking our computers or other peoples' computers
74
+* other activity which is illegal in the USA or your country
75
+
76
+And you agree to take reasonable precautions to maintain the security of your password and account by not sharing this information with other people.
77
+On our end, we will:
78
+
79
+* take reasonable precautions to maintain the security of the network, accounts and data
80
+* make occasional backups of account data (but don't count on it!)
81
+* attempt to maintain a high availability of the developer resources
82
+
83
+We reserve the right to close accounts at any time for any reason.
84
+
85
+Still interested? Great! We want to work with you, so please email `<bugs@sbgrid.org>` with your account request, and we'll get back to you as soon as possible.
stereo.md
... ...
@@ -0,0 +1,83 @@
1
+## General
2
+
3
+### 3D Stereo on RHEL/CentOS 6 systems
4
+
5
+Nvidia 3d stereo on RHEL 6 is reasonably straight-forward.
6
+Take a look [here](xorg_conf) for an example xorg.conf file. This configuration if for a Quadro 4000 card and a 1920x1080 display. Your settings may vary depending on your hardware.
7
+
8
+### 3D Stereo on RHEL/CentOS 7 systems with Gnome 3
9
+
10
+Nvidia 3D stereo typically requires disabling the Composite extension in X.org. Unfortunately, this extension is required to run Gnome 3 and the gdm login mangager supplied as the default desktop in CentOS and many other linux distributions.
11
+
12
+As of version 337.19 of the nvidia driver, some Quadro configurations can operate with the composite extension enabled, but stereo functionality depends on a stereo-aware composite manager. The Composite manager in CentOS7 Gnome3 is not stereo aware and 3D stereo does not work by default.
13
+
14
+In order to use 3D stereo on Gnome3 systems, you can switch from Gnome3/gdm to MATE/lightdm.
15
+The MATE Desktop Environment is the continuation of GNOME 2 which is the default desktop in CentOS 5 and 6.
16
+
17
+To change to MATE on CentOS7 systems, first, install the Extra Packages for Enterprise Linux (EPEL) repo:
18
+
19
+
20
+ yum install epel-release
21
+
22
+
23
+Then install the nvidia drivers from epel
24
+
25
+
26
+ yum install kmod-nvidia
27
+
28
+
29
+You may find you need these packages as well
30
+
31
+
32
+ yum install mesa-libGLU-9.0.0-4.el7.x86_64 libXScrnSaver-1.2.2-6.1.el7.x86_64
33
+
34
+
35
+Next, install lightdm which doesn't require the composite extension.
36
+
37
+
38
+ yum install lightdm
39
+
40
+
41
+From there we do a couple of groupinstalls
42
+You may have these rpms installed already, but just for good measure
43
+
44
+
45
+ yum group install "X Window System"
46
+
47
+
48
+then
49
+
50
+
51
+ yum groupinstall "MATE Desktop"
52
+
53
+
54
+Toggle from gdm to lightdm in systemd using systemctl.
55
+To see what you are using as a 'display manager', check this link
56
+
57
+
58
+ ls -l /etc/systemd/system/display-manager.service
59
+
60
+
61
+This is likely pointing to gdm and should be changed to point to lightdm.
62
+You can change that with systemctl.
63
+
64
+
65
+ systemctl disable gdm
66
+ systemctl enable lightdm
67
+
68
+
69
+And then this will reload you changes, similar to changing runlevels with sysinitV
70
+
71
+
72
+ systemctl isolate graphical.target
73
+
74
+
75
+Make sure to select MATE from the drop menu as your graphical before you login.
76
+## Mac OS X
77
+
78
+* Prior to OS X 10.8, the stock X11 has better compatibility than the Xquartz development version available from MacOSForge. With the OS X 10.8 release, X11 is no longer included by Apple, and you'll need to install the latest Xquartz version offered on the MacOSForge site to run X11-based applications.
79
+
80
+## Laptops
81
+
82
+* You can create a local installation to take along with you.
83
+http://www.sbgrid.org/wiki/laptopinstall
versions.md
... ...
@@ -0,0 +1,71 @@
1
+# Overriding Software Versions
2
+
3
+Most of the software titles installed in the SBGrid software collection have several versions available. We try to provide at least three versions of each software package, although very old or very new applications may have fewer versions.
4
+
5
+## Display Available Versions
6
+
7
+To list the available versions of a software title, you can use the *sbgrid* utility. Running *sbgrid -l* will list the versions for your current platform. From a Linux machine, a search for CCP4 versions looks like this:
8
+
9
+
10
+ $ sbgrid -l ccp4
11
+ Version information for: /programs/x86_64-linux/ccp4
12
+
13
+ Default version: 7.0
14
+ In-use version: 6.5
15
+ Other available versions: 6.4.0 6.3.0
16
+ Overrides use this shell variable: CCP4_X
17
+
18
+
19
+Running *sbgrid -L* will print the available versions for the three current platforms (OS X PowerPC and Intel, Linux).
20
+
21
+
22
+ $ sbgrid -L gnuplot
23
+ Version information for: /programs/x86_64-linux/gnuplot
24
+
25
+ Default version: 4.6.4
26
+ In-use version: 4.6.4
27
+ Other available versions: 4.6.0 4.4.3 4.2.6
28
+ Overrides use this shell variable: GNUPLOT_X
29
+
30
+ Version information for: /programs/i386-linux/gnuplot
31
+
32
+ Default version: 4.6.0
33
+ Other available versions: 4.4.3 4.2.6
34
+ Overrides use this shell variable: GNUPLOT_L
35
+
36
+ Version information for: /programs/i386-mac/gnuplot
37
+
38
+ Default version: 4.6.4
39
+ Other available versions: 4.4.3 4.2.6 4.6.0
40
+ Overrides use this shell variable: GNUPLOT_M
41
+
42
+
43
+## Overriding Default Versions
44
+
45
+To override the default version of a software title, you need to create an override file. The shell configuration scripts will read this file and override the default version.
46
+
47
+The override file is named .sbgrid.conf and should be placed in your $HOME directory. The syntax for overrides is a simple key=value pair. E.g. it should look like this:
48
+
49
+`APPLICATION_X=1.2`
50
+
51
+APPLICATION_X is the name of the application followed by an underscore and the shorthand initial for the architecture. The last field on the line is the software version number you want to use. The initials for each software branch are:
52
+
53
+| Software Directory | Shorthand Initial |
54
+| ------------------ | ----------------- |
55
+| i386-linux | _L |
56
+| x86_64-linux | _X |
57
+| i386-mac | _M |
58
+
59
+To quickly open the overrides file in your default $EDITOR, you can run *sbgrid-overrides -e*.
60
+
61
+When you open a new shell, you will see a message about the overridden version. Note that a site administrator can create sitewide or machine-specific overrides as well.
62
+
63
+The example output below shows CCP4 on linux being changed from the default version:
64
+
65
+ Configuration options in effect for this shell:
66
+ Variable Setting Configuration
67
+ CCP4_M 6.3.0 personal
68
+
69
+
70
+## Disabling a Software Package
71
+To disable a specific software package, set its version to "disable" in your ~/.sbgrid.conf file.
workstation_setup.md
... ...
@@ -0,0 +1,35 @@
1
+## General
2
+
3
+* You should use a network installation for workstations and servers if at all possible. NFS, SMB and AFP all work for sharing a single installation among multiple machines. The decreased admin overhead, bandwidth and disk savings of sharing a single installation is a significant benefit.
4
+
5
+## Linux
6
+Information on configuring 3D stereo can be found [here](stereo)
7
+
8
+* All Linux machines need installations of a recent version (6.17+) of csh/tcsh. Red Hat Enterprise Linux 6 and other recent distributions usually have an up to date version. There are RPMs for Red Hat/CentOS/Scientific Linux 5.x in the software tree in the architecture specific directories:
9
+
10
+ /programs/i386-linux/system/RPMs
11
+ /programs/x86_64-linux/system/RPMs
12
+
13
+* SELinux must be disabled.
14
+
15
+* 64-bit machines need a number of 32-bit compatibility packages installed **even when using the 64-bit software branch**. This is due to a number of binary-only and legacy packages that are only available as 32-bit applications. Additional fonts for Tcl/Tk, Motif and Xview applications are also encouraged.
16
+
17
+Packages for RHEL 6, and 7 and compatible machines:
18
+
19
+ yum install glibc.i686 libgcc.i686 libX11.i686 libXaw.i686 libXext.i686 ncurses.i686 \
20
+ redhat-lsb.i686 xorg-x11-fonts-75dpi xorg-x11-fonts-100dpi xorg-x11-fonts-ISO8859-1-75dpi \
21
+ xorg-x11-fonts-ISO8859-1-100dpi xorg-x11-fonts-misc libXScrnSaver
22
+
23
+
24
+* The same 32-bit requirement applies for 64-bit Debian and Ubuntu machines. This is supported via the [Multiarch](https///wiki.debian.org/Multiarch/HOWTO) system. You'll need at least the following packages to run most of the software. Please email us if you discover we have left some out.
25
+`apt-get install libc6-i386 lsb tcsh gawk libjpeg62 curl libx11-6:i386 libxext6:i386`
26
+
27
+## Mac OS X
28
+Information on configuring 3D stereo can be found [here](stereo)
29
+
30
+* Prior to OS X 10.8, the stock X11 has better compatibility than the Xquartz development version available from MacOSForge. With the OS X 10.8 release, X11 is no longer included by Apple, and you'll need to install the latest Xquartz version offered on the MacOSForge site to run X11-based applications.
31
+
32
+## Laptops
33
+
34
+* You can create a local installation to take along with you.
35
+http://www.sbgrid.org/wiki/laptopinstall