After you have created your CyVerse account and been granted access to the visual interactive computing environment (VICE) portion of the Discovery Environment data science workbench, you’ll be able to start a GUI based app.
Figure: More than you wanted to know about how this stuff works.
The Data Store¶
The CyVerse Data Store uses iRODS as a cloud storage system. When you see the data in the browser, it looks like a conventional file tree with folders and filenames.
CyVerse started out with the project name “iPlant Collaborative”, and our data store still retains the
iplant zone name in iRODS.
Windows vs Linux
If you’re a Windows user, you’re used to your file path looking something like this:C:\Documents\Folder Name\File Name.pdf
The volume or drive is assigned a letter, e.g.
C:\, and there may be spaces in the folders and file names.
In Linux, the folder paths use a forward slash
/, and do not add a letter to the root volume name. Spaces in folders and file names are highly discouraged and are unusable on the command line as a space is a special character which separates arguments./home/username/documents/folder_name/file_name.pdf
Using special case styles like
kebab-casehelps to differentiate between words in folder and filenames.
1. Log into the Discovery Environment: https://de.cyverse.org
3. This opens a file explorer, in your personal username space
4. Create a new folder called
The new folder should be located:/iplant/home/username/NEON_Downloads
This folder is private, only you can see it.
5. Sharing a folder
Click on the ‘Share’ tab and ‘Share with Collaborators’ option.
Type in a user’s given name and it should be searched and pop up. You will not see their
username, only their identity and institution information.
You have three options in granting privileges to others:
readpermissions allows the users to see and download the files and folder
writepermissions allows the user to modify the file and folder name.
ownpermissions allows the user to modify the file and folder and the ability to create and delete
Type in ‘Public User’ – adding this user will share the directory with all other CyVerse users when they are authenticated.
Type in ‘Anonymous User’ – adding this user will share the directory with the open internet (it will become visible on the internet via https://data.cyverse.org/dav-anon/
6. Look into the Community Data folder
These are public folders that have been ‘shared’ with all CyVerse users or with the open internet (via the Anonymous User group):/iplant/home/shared/
Navigate to ‘NEON_workshop/’ and ‘data/’/iplant/home/shared/NEON_workshop/data
There are some sample NEON AOP Data in here that we’ll get to tomorrow.
There are many more Community Data folders in CyVerse that you cannot see – that’s because they have not been shared with the ‘Public’ or ‘Anonymous’ user groups.
You do not have
ownpermission on any Community folders, so you cannot change them.
7. Look into the ‘Shared with Me’ folder
These folders are private user accounts that have public data in them or have been shared with you personally.
8. Access the Data Store from Cyberduck (Windows and Mac OS X only)
9. Access the Data Store from WebDav (browser based)
In your browser, navigate to https://data.cyverse.org
WebDav is a read-only space for viewing data that are already in the data store
https://data.cyverse.org/dav/folder path requires authentication with your CyVerse username and password
https://data.cyverse.org/dav-anon/folder path is public and anonymous read only to anyone on the interent.
Where does your data live?
When you download data from the internet to your local computer they’re isolated. How do you share them back with your team?
These services are incredibly useful.
However, file storage and sharing platforms like Box and Drive were not designed for machine readability and rapid requests for many (i.e. thousands to millions) of requests by anonymous users or even by trusted users. (see Google Drive vs Google Cloud for an explanation)
Conventional file services like
ftp://(file transfer protocol), function over HTTP and HTTPS. The same is true for Amazon Web Services
s3storage object buckets. (S3 explained)
How to work with your data in CyVerse
Downloading data from commercial cloud storage providors directly into CyVerse Data Store requires you have a running instance (virtual machine, or container in Discovery Environment) where the data can be staged before moving them onto the Data Store.
Uploading data to CyVerse is dependent upon your local internet service provider.
The App Catalog¶
If you signed up for the workshop, you will have already been added to the NEON Community group. We have added a couple of apps that have all of the tools needed for the workshop.
These Apps are yours to use! You can install new packages and software into them, but if that becomes too time consuming, consider learning about how to integrate your own Tools and Apps using the Apps and Tool Guide.
App – a graphical interface for starting a “Tool” here in the Discovery Environment. The App window can be customized to use any set of conditionals, parameters, resource requirements, input data, or output folders needed to do your analysis. An App can be “interactive” like the RStudio or Jupyter Lab, “executable” like a command line script, or “OSG” for high throughput parallel computing on the Open Science Grid.
Tool – a “Tool” is a Docker container which has been added to the Discovery Environment tool manager. It must be public on the Docker Hub or another Docker Registry (e.g. quay.io, NVIDIA NGC, etc.). After the tool manager template has been completed, the container will be added to the Discovery Environment. Click the “Manage Tools” with the wrench icon in the Apps window, to add your containers. After the Tool is integrated a private App can be created.
10. Under Categories and “My Apps” and “My Communities” you should see a group called NEON
11. In the group are a couple of shared apps with the |beta| tag.
Description of output and results
You should now understand the basics of the Classic Discovery Environment Interface.
- Data Store
Fix or improve this documentation
- Search for an answer: CyVerse Learning Center
- Ask us for help: click on the lower right-hand side of the page
- Report an issue or submit a change: Github Repo Link
- Send feedback: learning@CyVerse.org