Gold User's Guide

Scott Jackson

Pacific Northwest National Laboratory

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

  • Neither the name of the Battelle nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


Table of Contents
Notice
1. Overview
1.1. Background
1.2. Features
1.3. Interfaces
1.3.1. Command Line Clients
1.3.2. Interactive Control Program
1.3.3. Web-based Graphical User Interface
1.3.4. Perl API
1.3.5. SSSRMAP Wire Protocol
2. Installation
2.1. Preparation
2.1.1. Select a Database
2.2. Install Prerequisites
2.2.1. PostgreSQL database 7.2 or higher (or other tested database) [REQUIRED]
2.2.2. Perl 5.6.1 or higher (with suidperl) [REQUIRED]
2.2.3. libxml2 2.4.25 or higher [REQUIRED]
2.2.4. Gnu readline 2.0 or higher [OPTIONAL]
2.2.5. Apache Httpd Server 2.0 or higher [OPTIONAL]
2.2.6. OpenSSL 0.9.5a or higher [OPTIONAL]
2.2.7. mod_ssl 2.26 or higher [OPTIONAL]
2.3. Configuration
2.4. Compilation
2.5. Perl Module Dependencies
2.6. Installation
2.7. General Setup
2.8. Database Setup
2.9. Web Server Setup
2.10. Bootstrap
2.11. Startup
2.12. Initialization
3. Getting Started
3.1. Define Users
3.2. Define Machines
3.3. Define Projects
3.4. Add Users to the Projects
3.5. Make Deposits
3.6. Check The Balance
3.7. Integrate Gold with your Resource Management System
3.8. Obtain A Job Quote
3.9. Make A Job Reservation
3.10. Charge for a Job
3.11. Refund a Job
3.12. List Transactions
3.13. Examine Account Statement
3.14. Examine Project Usage
4. Getting More Advanced
4.1. Define Projects
4.2. Define Accounts
4.3. Make Deposits
4.4. Check The Balance
4.5. Define Charge Rates
4.6. Obtain A Guaranteed Job Quote
4.7. Make A Quoted Job Reservation
4.8. Charge for a Quoted Job
4.9. Partially Refund a Job
4.10. Examine Account Statement
5. Managing Users
5.1. Creating Users
5.2. Querying Users
5.3. Modifying Users
5.4. Deleting Users
6. Managing Machines
6.1. Creating Machines
6.2. Querying Machines
6.3. Modifying Machines
6.4. Deleting Machines
7. Managing Projects
7.1. Creating Projects
7.2. Querying Projects
7.3. Modifying Projects
7.4. Deleting Projects
7.5. Project Usage Summary
8. Managing Accounts
8.1. Creating Accounts
8.2. Querying Accounts
8.3. Modifying Accounts
8.4. Making Deposits
8.5. Querying The Balance
8.6. Personal Balance
8.7. Making Withdrawals
8.8. Making Transfers
8.9. Obtaining an Account Statement
8.10. Deleting Accounts
9. Managing Allocations
9.1. Creating Allocations
9.2. Querying Allocations
9.3. Modifying Allocations
9.4. Deleting Allocations
10. Managing Reservations
10.1. Creating Reservations
10.2. Querying Reservations
10.3. Modifying Reservations
10.4. Deleting Reservations
11. Managing Quotations
11.1. Creating Quotations
11.2. Querying Quotations
11.3. Modifying Quotations
11.4. Deleting Quotations
12. Managing Jobs
12.1. Creating Jobs
12.2. Querying Jobs
12.3. Modifying Jobs
12.4. Deleting Jobs
12.5. Obtaining Job Quotes
12.6. Making Job Reservations
12.7. Charging Jobs
12.8. Issuing Job Refunds
13. Managing Charge Rates
13.1. Creating ChargeRates
13.2. Querying ChargeRates
13.3. Modifying Charge Rates
13.4. Deleting Charge Rates
14. Managing Transactions
14.1. Querying Transactions
15. Managing Roles
15.1. Querying Roles
15.2. Querying Role Users
15.3. Querying Role Actions
15.4. Creating Roles
15.5. Associating an Action with a Role
15.6. Adding a Role to a User
15.7. Removing an Action from a Role
15.8. Removing a Role from a User
15.9. Deleting Roles
16. Managing Passwords
16.1. Creating Passwords
16.2. Querying Passwords
16.3. Modifying Passwords
16.4. Deleting Passwords
17. Using the Gold Shell (goldsh)
17.1. Usage
17.2. Command Syntax
17.3. Valid Objects
17.4. Valid Actions for an Object
17.5. Valid Predicates for an Object and Action
17.6. Common Options
17.7. Common Actions Available for most Objects
17.7.1. Query Action
17.7.2. Create Action
17.7.3. Modify Action
17.7.4. Delete Action
17.7.5. Undelete Action
17.8. Multi-Object Queries
18. Customizing Gold Objects
18.1. Removing an Attribute from an Object
18.2. Adding an Attribute to an Object
18.3. Modifying an Attribute
18.4. Creating a Custom Object
18.5. Adding an Action to an Object
18.6. Examples Creating Custom Objects
19. Integration with the Resource Management System
19.1. Dynamic versus Delayed Accounting
19.1.1. Delayed Accounting
19.1.2. Dynamic Accounting
19.2. Interaction Points
19.2.1. Job Quotation @ Job Submission Time [Optional — Recommended]
19.2.2. Job Reservation @ Job Start Time [Optional — Highly Recommended]
19.2.3. Job Charge @ Job End Time [Required]
19.3. Methods of interacting with Gold
19.3.1. Configuring an application that already has hooks for Gold
19.3.2. Using the appropriate command-line client
19.3.3. Using the Gold control program
19.3.4. Use the Perl API
19.3.5. Communicating via the SSSRMAP Protocol
20. Configuration Files
20.1. Server Configuration
20.2. Client Configuration
List of Examples
1-1. Listing Users
1-2. Listing Users
1-3. Listing Users
1-4. Listing Users
1-5. Listing Users
3-1. Let's add the users amy, bob and dave.
3-2. Let's define machines called colony and blue.
3-3. We will define the projects biology and chemistry.
3-4. Adding users to our projects.
3-5. Let's add 360000000 credits to each project. We will cause them both to be valid just for the fiscal year 2005.
3-6. Let's look at amy's balance
3-7. You may just want the total balance for a certain project and machine
3-8. We'll assume our job has the following characteristics:
3-9. Let's see how much it will cost to run our job.
3-10. Make a reservation for our job.
3-11. Issue the charge for our job.
3-12. Let's isse a refund for our job.
3-13. Let's list all the job transactions
3-14. It may also be illustrative to examine what transactions actually composed our charge request...
3-15. We can request an itemized account statement over all time for the chemistry project (account 2)
3-16. Display usage by user for the chemistry project
4-1. Now we will define the projects. This time we will define the project members at the same time.
4-2. We will create some accounts for use by the biology and chemistry projects.
4-3. Let's deposit 100 million credits for use by the biology project. We are going to establish a use-it-or-lose-it policy here in which one fourth of the credits expire each quarter. Since there is only one account for the biology project, we can specify the project name in the deposit.
4-4. Next we will make some deposits valid toward the chemistry project for the entire year. Since there are multiple accounts for the chemistry project, we must specify the appropriate account id in the deposit.
4-5. We can now take a closer look at the accounts and the allocations that we have created.
4-6. Let's look at amy's balance
4-7. Let's just get amy's balance for chemistry on colony.
4-8. Now let's just get the total that can be used by amy for chemistry on colony. This includes amy's available credit.
4-9. Let's examine the predefined charge rates.
4-10. Let's say we want to charge for memory used
4-11. We also want a quality of service multiplier
4-12. Creating another quality-based charge multiplier
4-13. Let's take a look at the current charge rates.
4-14. Let's request a guaranteed charge quote that reflects the memory and quality of service we expect to use.
4-15. Make a reservation for our job that reflects our resource and quality preferences while specifying the quote id.
4-16. Let's change a charge rate and issue the charge for our job. We will request that the quote be honored.
4-17. Suppose you want to issue a partial refund.
4-18. We can request an itemized account statement over all time for account 3 (chemistry for amy)
5-1. Creating a user
5-2. Listing all info about active users
5-3. Displaying bob's phone number
5-4. Listing all user names without the header
5-5. Listing a user's projects
5-6. Activating a user
5-7. Changing a user's email address
5-8. Deleting a user
6-1. Creating a machine
6-2. Listing all inactive machine names and descriptions
6-3. Deactivating a machine
6-4. Deleting a machine
7-1. Creating a project
7-2. Creating a project and specifying user members at the same time
7-3. Listing all info about all projects
7-4. Displaying the name and user members of a project in long format
7-5. Listing all project names
7-6. Deactivating a project
7-7. Adding users as members of a project
7-8. Adding machines as members of a project
7-9. Deleting a project
7-10. Displaying a usage summary for the chemistry project during the third quarter of 2006
8-1. Creating an account
8-2. Creating a wide-open account
8-3. Creating an account valid toward all biology project members except for dave and all machines except for blue
8-4. Listing all info about all accounts with multi-valued fields displayed in a multi-line format
8-5. Listing all info about all accounts useable by dave
8-6. Adding a user to the list of users that share the account
8-7. Making a deposit
8-8. Making a deposit "into" a project
8-9. Creating a credit allocation
8-10. Querying the project balance detail broken down by account
8-11. Querying the total balance for a particular user in a particular project on a particular machine
8-12. List the projects and available balance amy can charge to
8-13. List my (project) balances
8-14. List my balance in (Processor) hours
8-15. Making a withdrawal
8-16. Making a withdrawal "from" a project
8-17. Transferring credits between two accounts
8-18. Transferring credits between two single-account projects
8-19. Generating an account statement for the third quarter of 2006
8-20. Deleting an account
9-1. Listing allocations for account 4
9-2. Changing the end time for an allocation
9-3. Changing the credit limit for an allocation
9-4. Deleting an allocation
9-5. Purging inactive allocations
10-1. Listing all info about all reservations for bob
10-2. Listing all info about all reservations that impinge against amy's balance
10-3. Changing the expiration time of a reservation
10-4. Deleting a reservation by name (JobId)
10-5. Deleting a reservation by ReservationId
10-6. Purging stale reservations
11-1. Listing all info about all quotes for user amy on machine colony
11-2. Changing the expiration time of a quotation
11-3. Deleting a quotation
11-4. Purging stale quotations
12-1. Creating a job record
12-2. Show specific info about jobs run by amy
12-3. Changing a job
12-4. Deleting a job
12-5. Requesting a quotation
12-6. Requesting a guaranteed quote
12-7. Creating a reservation
12-8. Issuing a job charge
12-9. Issuing a job refund
13-1. Creating a consumable resource charge rate
13-2. Creating another consumable resource charge rate
13-3. Creating a usage charge rate
13-4. Creating another usage charge rate
13-5. Creating a name-based multiplier rate
13-6. Creating another name-based multiplier rate
13-7. Creating a value-based multiplier rate
13-8. Listing all charge rates
13-9. Changing a charge rate
13-10. Deleting a charge rate
14-1. List all deposits made in 2004
14-2. List everything done by amy since the beginning of 2004
14-3. List all transactions affecting Job Id PBS.1234.0
14-4. List all transactions affecting charge rates
15-1. Listing all roles
15-2. Listing all role users
15-3. Listing all role actions
15-4. Creating a Manager role
15-5. Allow the Manager to change role responsibilities
15-6. Adding a user to the Manager role
15-7. Don't let UserServices Create or Update Projects
15-8. Removing dave as a Manager
15-9. Deleting the Manager role
16-1. Creating a password
16-2. List the users who have set passwords
16-3. Changing amy's password
16-4. Deleting a password
17-1. Specifying the command as direct arguments
17-2. Using the interactive prompt
17-3. Reading commands from a file
17-4. Listing all objects
17-5. Listing all actions associated with the Account object
17-6. Show the usage for Allocation Query
17-7. Return the number of inactive reservations
17-8. Add a new project member
17-9. Change/set scottmo phone number and email address
17-10. Extend all reservations against project chemistry by 10 days
17-11. Get rid of the pesky Jacksons
17-12. Let's resurrect the deleted users that were active
17-13. Print the current and total allocation summed by project
17-14. Show all active projects for amy or bob
18-1. Removing the Organization attribute from Machine
18-2. Perhaps we don't care to track the Executable attribute in a Job
18-3. Adding a Country Attribute to User
18-4. We need to track submission time in Jobs
18-5. Change User Organization values to not be restricted by foreign key
18-6. Creating a Node Object
18-7. We need to track submission time in Jobs
18-8. Adding a Modify Action to Transaction
18-9. Creating a License object to track license usage and charges.
18-10. Using Gold as a Grid Map File.
19-1. Configuring Maui to use Gold
19-2. To issue a charge at the completion of a job, you would use gcharge:
19-3. To issue a charge you must invoke the Charge action on the Job object:
19-4. To make a charge via this interface you might do something like:
19-5. The message might look something like: