The sqlr-start program is provided to simplify the startup process.
sqlr-start -id ID -config CONFIGFILE [-localstatedir LOCALSTATEDIR]
The sqlr-start program starts up one instance of sqlr-listener, one instance of sqlr-scaler and the number of sqlr-connection daemons specified in the config file for the given id.
The sqlr-start program also checks to see if a sqlr-cachemanager is running. If one is then it leaves it running. If not then it starts one.
See the section about sqlr-cachemanager below for an important note about it.
If the optional LOCALSTATEDIR argument is used, it is passed to the programs started by sqlr-start as appropriate.
Debugging and TracingThe instance tag of the CONFIGFILE contains an optional debug attribute. When it is set, the sqlr-start program starts up debugging versions of the sqlr-listener and/or sqlr-connection daemons. These daemons output debugging information to files in /usr/local/firstworks/var/sqlrelay/debug. See Configuring SQL Relay for more information on the debug attribute.
The sqlr-start program also takes an optional -strace parameter. When run as root, if the -strace parameter is used, sqlr-start starts the sqlr-connection daemons using strace -ff -o sqlr-connection-strace causing sqlr-connection-strace.pid files to be written to the root directory. See the strace man page for more information.
Starting Daemons IndividuallyIf for some reason you don't want to use sqlr-start, you can start the daemons individually.
Setting Up a ListenerThe first step in is setting up a listener. The job of the listener is to connect to a TCP port and wait for queries. When it gets one, it waits for an available connection daemon and hands off the client to it.
Run the sqlr-listener daemon to establish a listener.
sqlr-listener -id ID -config CONFIGFILE [-localstatedir LOCALSTATEDIR]
CONFIGFILE is the filename of the configuration file to use. The default configuration file is /usr/local/firstworks/etc/sqlrelay.conf
ID is an identifier that associates a listener with a pool of connections.
Only one listener needs to be established for a given ID.
LOCALSTATEDIR is the base directory for temporary files and debug logs. By default, this is /usr/local/firstworks/var, PREFIX/var if you compiled SQL Relay with a prefix other than /usr/local/firstworks or possibly another directory if you specified a different localstatedir when compiling SQL Relay. Under the localstatedir, temporary files go in sqlrelay/tmp and debug logs go in sqlrelay/debug.
Setting Up a ConnectionConnecting to a database is the next step. To estabish a connection, run one or more of the sqlr-connection daemons.
sqlr-connection-"dbase" -id ID -config CONFIGFILE -connectionid CONNECTIONID [-localstatedir LOCALSTATEDIR]
Where "dbase" is one of oracle8, mysql, postgresql, sqlite, odbc, db2, firebird, sybase, freetds or mdbtools.
CONFIGFILE is the configuration file to use when starting the daemon. The default configuration file is /usr/local/firstworks/etc/sqlrelay.conf
ID is just an identifier. Starting multiple connections with the same ID make them available as a pool of connections. The more connections you start, the more queries can be executed in parallel. In addition, the ID argument specifies which config file entry to use.
CONNECTIONID is another identifier. When using replicated or clustered databases, within a pool of connections, some connections will connected to 1 machine and some will connect to another. This argument tells the connection daemon which machine to connect to. When not using clustered or replicated databases, this id will be the same for every connection in the pool.
LOCALSTATEDIR is the base directory for temporary files and debug logs. By default, this is /usr/local/firstworks/var, PREFIX/var if you compiled SQL Relay with a prefix other than /usr/local/firstworks or possibly another directory if you specified a different localstatedir when compiling SQL Relay. Under the localstatedir, temporary files go in sqlrelay/tmp and debug logs go in sqlrelay/debug.
Setting Up the ScalerThe third step is running the scaler. You can start as many sqlr-connection daemons as you like. If you start fewer than the maximum number defined for the ID, the sqlr-scaler daemon will fire up new connections on demand. The new connections will time out after a period of inactivity and shut down on their own.
Run the sqlr-scaler daemon to establish a scaler.
sqlr-scaler -id ID -config CONFIGFILE
CONFIGFILE is the filename of the configuration file to use. The default configuration file is /usr/local/firstworks/etc/sqlrelay.conf
ID is the same as the ID argument in the sqlr-connection daemon and associates a scaler with a pool of connections.
Only one scaler needs to be established for a given ID.
Setting Up the Cache ManagerThe fourth step is running the cache manager. Clients can cache result sets with a time-to-live on them. The cache manager goes through the cached result sets periodically and removes the ones that have expired.
Run the sqlr-cachemanager daemon to establish the cache manager.
sqlr-cachemanager [-scaninterval INTERVAL] [-cachedirs CACHEDIRS]
The optional INTERVAL argument specifies in seconds how often the cache manager scans the result sets. Each scan is scheduled from the end of the previous scan. If the argument is left off, the cache manager scans at a default interval of 30 seconds.
The optional CACHEDIRS argument is a colon seperated list of directories to scan for cache files. If you use sqlr-start and specify a LOCALSTATEDIR, sqlr-start will pass the -scandirs LOCALSTATEDIR/sqlrelay/cache to sqlr-cachemanager.
Only one cache manager needs to be started per machine.
Important Note: Since cache managers clean up after SQL Relay clients, not servers, they need to be run on machines which run clients that could cache result sets. These may not be the same machines that run the sqlr-listener and sqlr-connection daemons. If there is no SQL Relay installation on those machines, you should create the cache directory (/usr/local/firstworks/var/sqlrelay/cache unless changed at compile time), install the sqlr-cachemanager program by itself and set it up to run at boot time.
Extending, Killing and RestartingOnce the daemons are up and running, clients can use them. In the event that more connections are required, additional sqlr-connection-"dbase" daemons may be started from the command line using the same ID and CONFIGFILE and will be immediately available to clients.
Connection daemons should not be killed once they have been started without restarting everything with that ID.
The sqlr-stop command is provided to kill running daemons. It accepts two optional arguments: the id of the instance to kill and SIGKILL.
sqlr-stop [id] [SIGKILL]
Running it kills anything with "sqlr-" and the supplied id in its name. It uses grep, so it's not perfect. If you have id's like "web" and "webster" and run "sqlr-stop web" then it will kill both of them. Running it with no arguments will kill all SQL Relay processes.
Supplying sqlr-stop with the SIGKILL option will cause it to pass the -9 option to the kill command. It is not possible to supply the SIGKILL option without supplying an id. Note that when the SIGKILL option is used, semaphores and shared memory segments will be left lying around and will need to be cleaned up. Use the ipcs and ipcrm commands to clean them up.
Using the Command Line InterfacesFour command line utilities are provided for use with SQL Relay: query, fields, backupschema and sqlrsh. The syntax for each is:
query HOST PORT SOCKET USER PASSWORD QUERY
fields HOST PORT SOCKET USER PASSWORD TABLE
backupschema HOST SOCKET PORT USER PASSWORD
sqlrsh HOST PORT SOCKET USER PASSWORD
sqlr-export HOST PORT SOCKET USER PASSWORD (table|sequence) tablename > exportfile.xml
sqlr-import HOST PORT SOCKET USER PASSWORD exportfile.xml
or, if there's an sqlrelay.conf file that the client can read:
query -id ID QUERY
fields -id ID TABLE
sqlrsh -id ID
sqlr-export -id ID (table|sequence) tablename > exportfile.xml
sqlr-import -id ID exportfile.xml
The HOST, PORT, SOCKET, USER and PASSWORD arguments specify which listener to connect to and what authentication to use.
The ID argument refers to an entry in the sqlrelay.conf file from which host, port, socket, user and passwords can be read.
The query command executes the query specified in the QUERY parameter and returns the result set as a quote/comma/return seperated value list to standard output.
The fields command returns a comma seperated list of the fields in the table specified by the TABLE argument to standard output.
The backupschema command only works against oracle databases and uses the "select table_name from user_tables" query in combination with the query and fields commands to generate Oracle SQL Loader control files for all tables owned by the user a particular connection is logged in as. The command is useful for backing up an entire schema of data to a readily re-importable format.
sqlrsh is an interactive query tool similar to (though not as powerful as) SQL Plus, mysql, psql, isql and the like. Run it and type help; at the prompt for detailed usage instructions. When sqlrsh starts up, it reads and executes two rc files, the system rc file (most likely /usr/local/firstworks/etc/sqlrshrc) and a personal rc file .sqlrshrc in your home directory. These files should contain sqlrsh commands, ending in semicolons, seperated by carraige returns. Any commands may be used in the rc files, even queries.
The sqlr-export command exports the specified table or sequence to a file from which sqlr-import can later re-import it.
The sqlr-import command reads the specified file and imports the data contained in the file into the table or sequence specified in the file. Note, that sqlr-import does not create tables or sequences, it just imports data into an already-existing table or sequence.
sqlr-export and sqlr-import may be used to export data from one database and import it into another database, even if the databases are dissimilar (eg. one is PostgreSQL and the other is MySQL), if the structures of the tables in both databases are compatible.
Using The Status MonitorThe sqlr-status program displays statistics about a running SQL Relay instance. You can run it as follows, replacing instance with the name of the SQL Relay instance that you would like to read statistics from:
sqlr-status -id instance
Currently it generates output like:
Open Server Connections: 3 Opened Server Connections: 3 Open Client Connections: 0 Opened Client Connections: 1 Open Server Cursors: 15 Opened Server Cursors: 16 Times New Cursor Used: 0 Times Cursor Reused: 10 Total Queries: 10 Total Errors: 1
Here are descriptions of the statistics:
General difficulty with FreeTDS or Sybase connections.
Sybase and FreeTDS both provide the libct.so library and the ctpublic.h header file. If FreeTDS is installed from an RPM or other package, it is possible for its libct.so to be installed in /usr/lib and its header file to be installed in /usr/include. This can cause the Sybase connection to be compiled against ctpublic.h and linked against libct.so from FreeTDS rather than from Sybase.
Alternatively, if the Sybase header and library end up in those locations (ie. if they were manually copied there) then the FreeTDS connection could be compiled or linked against them.
Generally, the solution is to install FreeTDS somewhere other than /usr and omit the FreeTDS and Sybase library paths from /etc/ld.so.conf, /etc/ld.so.conf.d and LD_LIBRARY_PATH. By default, the build uses rpath's to cause the connections to look for libraries in the exact place that they were found at compile time, causing each connection to find the right library. This option can be turned off though. In that case, LD_LIBRARY_PATH should be set before running sqlr-start to assure that the connection dynamically links against the proper library. To see which libraries the connection will link against, run "ldd `which sqlr-connection-freetds`" or "ldd `which sqlr-connection-sybase`"
The configure script displays a warning that should encourage people to exercise care when compiling FreeTDS and Sybase connections. However, even if each connection is compiled against the proper header file, it's possible for either connection to dynamically link against the wrong library at run time.
Hopefully, one day, FreeTDS will support everything that the native Sybase libraries support and there will simply be an option to link the Sybase connection against one or the other.
FreeTDS or Sybase connection fails with error: Connection refused.
Sybase ASE 11.9.2 runs on port 4100 by default and MS SQL Server runs on port 1433 by default. If you try to connect to an instance of one of them and get an error, you can either reconfigure the instance to run on port 4000 or set the port argument of the connectstring in CONFIGFILE to the port that the instance is actually running on.
bind failed
The most common cause of this problem is configuring SQL Relay to listen on the same port that the database is listening on. For example, if your database is listening on port 4100 and you run SQL Relay on the same machine, you can't configure SQL Relay to listen on port 4100 or it will issue "bind failed" when the listener tries to run.
A slightly less common cause of this problem is configuring SQL Relay to listen on a port that some other service is already listening on. For example, web and cache servers often listen on port 8080 and IRC servers often listen on port 7000. You can see if some other service is listening on the port you want to SQL Relay to listen on by running netstat -ap | grep PORT where PORT is replaced with the port number that you'd like SQL Relay to listen on. If you get anything back from that command, then there is another service already listening on that port.
If you kill and restart the daemons and get the message: "bind failed." as the listener is starting, then there are 2 possibilities. First, all the daemons may not have been killed. In this case, kill them all and make sure they are dead by using ps -efal | grep sqlr- before restarting them. The second possibility is that the port the listener was listening on didn't get closed. Executing netstat -a | grep PORTNUMBER will reveal any connections still lingering on the port. If all the daemons are dead but the connections are still lingering, wait 2 minutes or so before restarting the daemons. The lingering connections should have timed out by then.
Everything starts fine but clients can't execute queries.
The most common cause of this problem is telling the client to connect to the port that the database is listening on rather than the port that SQL Relay is listening on. For example, if the database is listening on port 4100 and you have an instance of SQL Relay connected to it and have configured SQL Relay to listen on port 9000, then a common mistake would be to try to connect an SQL Relay client to port 4100 instead of port 9000.
SQL Relay daemons use semaphores and shared memory segments. If a daemon crashes unexpectedly, even if you kill all the other daemons, a semaphore or shared memory segment may still be hanging around. These can interfere with future attempts to start up daemons with the same ID. You can use the ipcs command to inspect the shared memory segments and semaphores and the ipcrm command to remove any lingering ones.
The SQL Relay daemons also use temporary files, usually located in /usr/local/firstworks/var/sqlrelay/tmp. That directory should have 777 permissions, but sometimes it doesn't. The sockseq file in that directory should have 666 permissions, but sometimes it doesn't. The files named ID and ID-CONNECTIONID owned by the user that started the connections in that directory should get removed by sqlr-stop, but sometimes they don't. Sometimes sqlr-stop isn't even used to stop the daemons.