Author: Nenad Rakocevic / SOFTINNOV Date: 23/09/2007 Documentation version: 1.6.1 Comments: mysql_at_softinnov.com
Table of Contents
1. QuickStart
2. Important notices
2.1. Changes in v1.2.1
2.2. Changes in v1.2
2.3. Changes in v1.1
2.4. Changes in v1.0
2.5. Changes in v0.9.9
2.6. Changes in v0.8.5 (for pre-0.8.5 users only)
3. Introduction
4. Setup
5. Connecting to the server
6. Sending commands
6.1. Send an SQL query
6.2. Send an SQL query inserting REBOL's values
6.3. Send server admin commands
7. Retrieving records
8. Closing connection
9. Compact format using READ
10. Shortcut format using SEND-SQL
11. Named Fields
12. Connection Properties
13. Types conversions
13.1. On Reading Records
13.2. On Inserting REBOL Values
13.3. User-defined conversion rules
14. Beta testing
15. Todo
16. Copyright
Installation:
Uncompress the ZIP archive on your local filesystem, then :Usage (square brackets contain optional information, unless it's specified otherwise) :>> do %mysql-protocol.r MySQL protocol loaded
orread mysql://[[user][:pass]@]host[:port]/[database[/table]]
orread/custom mysql://[[user][:pass]@]host[:port]/[database] ["...SQL query..."] (last brackets are mandatory)
ordb: open mysql://[[user][:pass]@]host[:port]/database ... send-sql[/flat][/named][/raw] db "SQL query or server command" ... close db
wheredb: open mysql://[[user][:pass]@]host[:port]/database ... insert db "SQL query or server command" copy[/part] db [rows [integer!]] or first db (basically, shortcut to : copy/part db 1) ... close db
user = a valid user name. pass = a valid password or '?. host = server address. (localhost or 127.0.0.1, if your server is local) port = server port. (default is 3306) database = a valid database name. table = a valid table name in selected database.
Date conversion has been slightly changed to cope with MySQL zero dates special values like : '0000-00-00' and '0000-00-00 00:00:00'. There are now converted by default, to the following REBOL values : 1-Jan-0000 or 1-Jan-0000/00:00.Note: These zero dates values are generated by MySQL when you try to insert invalid date or time values. You can also generate them by inserting a 0 value in a DATE or DATETIME field.
The behaviour and syntax of the READ function has changed in this release. The new syntax is :
Warning: this new READ syntax will break compatibility with previous versions.read mysql://root@localhost/ ==> return the list of databases read mysql://root@localhost/db ==> return the list of tables in database "db" read mysql://root@localhost/db/tbl ==> return the description of table "tbl" in database "db" read/custom mysql://root@localhost/db ["...sql query..."] ==> execute the sql query on database "db" and return the resultset.An init SQL string can now be specified for an opened port. That init string will be executed after automatic reconnections. (Useful to set, e.g., a specific charset value). Example :
db-port/locals/init: {SET NAMES 'latin2'}
There's now a new preferred way to query data, the SEND-SQL global function. The INSERT/COPY methods are still available for cases where finer-grained control or ressources optimization is required.
New named fields mode in the driver produce recordsets with named fields. Example :send-sql[/flat][/named][/raw] db "SQL query or server command"
Support for multiple statements in queries has been added (separated by semicolons). Now, you can, for example (%setup.sql containing a lot of SQL statements):author: send-sql/named db "SELECT id, name, year FROM authors WHERE id=1" print [author/name "is born in" author/year]
send-sql db read %setup.sql
Now MySQL servers v5 and up, are supported. The new passwords format is recommended for driver stability. If you're migrating your database from a 3.x version, please upgrade users passwords to new passwords encryption.New flat? property producing flat recordsets of only one block. See the "Connection Properties" section for more details.
This new version now handles all REBOL to SQL conversions. So, if you're using your own conversion functions, the new version can break your code or generate unwanted results (like double-escaping some characters). See the "Type conversion" section for more details.
0.8.5 is a big update which can break some scripts in some cases. Here are a list of the most important changes in the way the driver now behaves:
- MySQL errors are now thrown by INSERT. COPY and FIRST should only report connection lost or communication errors.
- A call to 'insert or 'close will automatically flush all unread pending data.
- When using 'copy/part or 'first, it's no more necessary to call them until you get 'none or []. When you read the last record, the end of the data stream is detected and handled internally.
- When sending an INSERT or UPDATE SQL command, now you don't need to call 'copy anymore. Just send your query and your port/locals/matched-rows value will be updated according to the number of rows affected by your query.
- If you only use the 'read function with this scheme, you don't need to change anything in your code ! :)
This driver allow REBOL developers to fully access the MYSQL RDBMS. It supports server protocols v9 and v10, so it should work with all versions of MySQL. (at least up to v5.0.21 which is the one i use for testing.)The driver is encapsulated into a REBOL scheme that will allow you to work with the database records as "external" series using INSERT, COPY, FIRST,... natives.
The driver public interface is close to the one proposed by REBOL/Command for various databases, but syntax and behaviour aren't strictly identical. Future MYSQL driver versions will be more compatible with the official REBOL databases API.
The driver should work correctly with all available REBOL products.
Unpack the ZIP archive and save the mysql-driver/ folder in your local filesystem. Installation done !To load the driver, just DO it, in console mode or in your scripts :
Now the driver is installed and ready to work.>> do %mysql-protocol.r MySQL protocol loadedYou can check at anytime if the driver has been loaded by doing:
>> probe first system/schemes [self default Finger Whois Daytime SMTP POP HTTP FTP NNTP MySQL] ^^^^^
To connect to a MySQL server, use the OPEN function :
wheredb: open mysql://[[user][:pass]@]host[:port]/database
user is a valid user name. pass is a valid password or ? (question mark). host is the server address. (localhost or 127.0.0.1, if your server is local) port is the server port. (use this option if the server isn't listening on the default port 3306) database is an existing database name. Example:
OPEN returns an opened port to the server which will be used for all subsequent actions.db: open mysql://root@localhost/mysql db: open mysql://dockimbel:[email protected]/test db: open mysql://127.0.0.1/billsIf the open process fails, it's most probably because you specified a wrong parameter or your user doesn't have the correct rights set. (check your server config, usually it's caused by an incorrect setup of your user's hosts rights).
If you want to avoid showing your password in the console to everyone, you can use the ? character as replacement. You will be asked to enter your password in a hidden field.
Example:
>> db: open mysql://dockimbel:?@localhost/test Password: *********
Use the INSERT function :
whereinsert port data
port is an opened port to a MySQL server. data is a string containing the SQL query. Example:
INSERT will return NONE if it's ok or else an error! value (which can be trapped with TRY).insert db "SELECT * FROM user" insert db "INSERT INTO mybooks VALUES ('Elan', 'REBOL Official Guide')" insert db {CREATE TABLE products ( name VARCHAR(100), version DECIMAL(2, 2), released DATE )}Note: Since version 1.1, the driver supports multiple statements in SQL queries. The statements must be separated by a ; (semicolumn). Examples :
insert db { DROP TABLE IF EXISTS products; CREATE TABLE products ( name VARCHAR(100), version DECIMAL(2, 2), released DATE ); INSERT INTO products VALUES ('cheyenne', '1.0', '2007-05-31'); INSERT INTO products VALUES ('mysql', '1.1', '2007-05-01'); }insert db read %setup.sql ;-- execute a big SQL file
Use the INSERT function :
whereinsert port [data word1 word2 ...]
port is an opened port to a MySQL server. data is a string containing the SQL query with '? characters. word1 word2 ... list of defined REBOL words or values All ? characters in the SQL query string will be replaced by the supplied REBOL values. If the number of ? characters is greater than the number of supplied values, the driver will send NULL values in place of the missing values. If the number of ? is lesser, the remaining REBOL values will be ignored.
Look at "Type conversion" to see how the driver handles REBOL to SQL conversions.
Example:
insert db ["INSERT INTO mybooks VALUES (?,?)" "Elan" "REBOL Official Guide"]author: "Elan" title: "REBOL Official Guide" insert db ["INSERT INTO mybooks VALUES (?,?)" author title]INSERT will return NONE if it's ok or else an error (which can be trapped with TRY).tables: [books movies cds games tmp] foreach table tables [ insert db ["DELETE FROM ?" table] ]
This feature is intended for developers who want to build an administration tool for MySQL. You should use these commands very carefully!Use the INSERT function :
whereinsert port [cmd arg1 arg2 ...]
port is an opened port to a MySQL server. cmd is a valid command name. arg1 arg2 ... list of command arguments List of valid commands with their arguments:
- quit : Close the current connection. Returns NONE.
- init-db "dbname" : Change the current working database. Returns NONE.
- create-db "dbname" : Creates a new database. Returns NONE.
- drop-db "dbname" : Destroy a database. Returns NONE.
- reload arg1 arg2 ... : Refresh or flush the server. Allowed arguments are :
grant Refresh grant tables log Start on new log file tables Close all tables hosts Flush host cache status Flush status variables threads Flush status variables slave Reset master info and restart slave thread master Remove all bin logs in the index and truncate the index
Example:
- shutdown : Shutdown the server ! Returns NONE.
- statistics : Get some stats from the server. Returns a string!.
- process-kill id : Kill a process in the server (id is a process id number). Returns NONE.
- debug : Activate the server debug mode. Returns NONE.
- ping : Ask if the server is alive. Returns TRUE if the server is alive or else NONE.
- change-user "user" "pass" "dbname" : Change the active user without closing the connection. Returns a NONE
Note: You should have the correct privileges set in the server to be allowed to use admin commands !insert db [init-db "test"] ; same as 'insert db "use test" print insert db [statistics] insert db [shutdown] insert db [reload grant] insert db [reload log grant status] ; for 'reload argument order doesn't matter.
To get some data from the server, you have to, first, send a query or a command with INSERT. Then, you can retrieve records with the following functions :
orcopy[/part] port [number]
wherefirst port
port is an opened port to a MySQL server. number is an integer, indicating the maximum number of records to retrieve. Records are returned in a 2-dimensional block array, that looks like this:
Watch out when choosing the method to get the records, they don't behave the same :; COL 1 COL 2... COL n [ [item1 item2 ... itemn] ; ROW 1 [... ... ... ... ] ; ROW 2 ... ; ... [... ... ... ... ] ; ROW m ]
copy Will return all the available records. The next call to COPY will return NONE. copy/part Will return the specified number of records. If you get less records than excepted, it's because you've reached the end of the recordset and there's no more record available. first Will return the next available record. If there's no more records, FIRST will return an empty block []. Note:
Example:
- If you don't read all the available records, they will be flushed on next call to INSERT or CLOSE.
- When dealing with big recordsets, you should care about memory usage. In this case just avoid using COPY ! Use FIRST or COPY/PART to get the records, especially if records contains blob binaries.
probe copy/part db 3 ; get 3 records probe first db ; get just 1 record probe copy db ; get all remaining recordsforeach row copy db [probe row] ; useful for a fast screen dumpwhile [not empty? row: first db][probe row] ; a memory saving version
Closing the port connection is done, as always, with the CLOSE function :
whereclose port
port is an opened port to a MySQL server. Don't forget to close the connection, even in a CGI script, because the server has allocated ressources for your connection and he has to free them. (Also, not closing a port will generate an error entry in the server logs !)
REBOL allows a compact format for getting data using an URL. The "miracle" is achieved by the READ function. READ can return informations from the server using the following syntax:
The result returned by READ will vary according to the URL ending part :read mysql://[[user][:pass]@]host[:port]/[database[/table]]
READ can also be used to execute an SQL query directly using the /CUSTOM refinement. The syntax is :
- / : returns the list of databases in this server (equivalent to SQL "SHOW DATABASES")
- /database : returns the list of tables for this database (equivalent to SQL "SHOW TABLES")
- /database/table : returns the table structure (equivalent to SQL "DESC table")
READ will open the connection to the specified MySQL server, send the given query, retrieve the resulting recordset and close the connection ! This all-in-one format can be useful in simple CGI scripts or for testing purpose from the console.read/custom mysql://[[user][:pass]@]host[:port][/database] ["...SQL query..."] (last brackets are mandatory)Note:
The syntax of READ has been changed in v1.2.0 breaking compatibility with previous versions.
Examples:
probe read mysql://root@localhost/ probe read mysql://root@localhost/mysql probe read mysql://root@localhost/mysql/user probe read/custom mysql://root@localhost/mysql ["SHOW databases"] probe read/custom mysql://root@localhost/mysql ["SHOW status"] probe read/custom mysql://root@localhost/mysql ["SELECT * FROM user"]; The most powerful 1-line script ! ;-) foreach row read/custom mysql://root@localhost/books ["SELECT * FROM authors"] [print row]
SEND-SQL is basically a shortcut to INSERT/COPY with handy refinements, it sends the query and returns the result :
wheresend-sql[/flat][/named][/raw] port data
port is an opened port to a MySQL server. data is a string or a block containing the SQL query. Refinements:
Refinement Description /flat same as flat? property, returns a flattened records block /named returns fields prefixed by their column names. (restricted use when combined with /flat) /raw same as auto-conv? property, returns all fields as string! values SEND-SQL supports all the features of the INSERT command, like prepared statements with REBOL values or server commands. This is the preferred and easier way to send queries in the general use case.
Note: There is a restriction using /flat with /named : in this case, the query must return only one record for the naming system to be able to work !
Example:
send-sql db "SELECT * FROM user" send-sql db ["INSERT INTO mybooks VALUES (?,?)" "Elan" "REBOL Official Guide"] print send-sql db [statistics] send-sql db read %setup.sqlforeach row send-sql db "SELECT * FROM authors" [print row]
By default, when retrieving records, you get directly the fields values. The only way then, to access the data is to use indexing :
If you want a more friendly access to values, you can activate the "named fields" mode using /named. The example becomes :send-sql/flat db "SELECT id, name, year FROM authors WHERE id=1" print [author/2 "is born in" author/3]
If you're using the INSERT/COPY combination, you can still get the columns names using the globally defined NAME-FIELDS function :author: send-sql/flat/named db "SELECT id, name, year FROM authors WHERE id=1" print [author/name "is born in" author/year]
wherename-fields port record
port is an opened port to a MySQL server. record is a block of resulting data (must represent only one record) So the example becomes then :
insert db "SELECT id, name, year FROM authors WHERE id=1" author: name-fields db first db print [author/name "is born in" author/year]
You can get more informations about the server or the result of a query, with the following object:
Here's a list of the currently available properties :port/locals ; port is an opened port to a MySQL server
Property Type Access Description max-rows integer! - (not yet supported, use COPY/PART instead) auto-conv? logic! read/write Set to ON will automatically convert types when decoding records. Set to OFF will produce records with all values of type string! auto-ping? logic! read/write Set to ON will allow automatic detection of lost connections and will try to reconnect if needed. It works by sending a 'ping command before each call to INSERT. Set to OFF will disable the automatic 'ping and reconnection system flat? logic! read/write Set to ON will produce 1-dimensional recordset (a flat block of all records). Set to OFF (default) will enclose each records in its own block (a block of block of records) init string! read/write init SQL commands to execute after an automatic reconnection matched-rows integer! read Will give the number of affected rows after an SQL INSERT or UPDATE query conv-list block! read Description list used for type conversion columns block! read List of columns objects for the last recordset retrieved protocol integer! read MySQL protocol version. (set by the server) version string! read Server version thread-id integer! read Thread ID for the current connection crypt-seed string! read Crypt seed used to encode passwords capabilities block! read List of server capabilities (word! values) error-code integer! read Last error code error-msg string! read Last error message Note:
- Just do a PROBE port/locals after sending a query to see an example. The properties not described here are reserved for driver's internal usage.
- Automatic reconnection can be very useful when you want to maintain a pool of connection with a MySQL server during a long time. With this feature on, your connections can survive to server restarts or connections closed by server on timeouts ! (by default, AUTO-PING? is set to ON)
The following table shows the default conversions applied to each value when reading records :
MySQL Type REBOL types decimal decimal! tiny integer! short integer! long integer! float decimal! double string! null none! (*) timestamp string! longlong string! int24 integer! date date! (**) time time! datetime date! (**) year integer! newdate string! var-char string! bit string! new-decimal string! enum string! set string! tiny-blob string! medium-blob string! long-blob string! blob string! var-string string! string string! geometry string! (*) MySQL 'NULL value is handled separately at low-level in the driver. 'NULL is always converted to NONE.
(**) MySQL zero dates values are converted to 1-Jan-0000 or 1-Jan-0000/00:00.
Changing the default conversion can be done with the CHANGE-TYPE-HANDLER function.
When you use the ? character in your SQL statement, the driver will automatically do some encoding for you on REBOL values. Here is the list of the built-in conversion rules :
REBOL Datatype SQL Encoding none! NULL date! 'yyyy-mm-jj' or 'yyyy-mm-jj hh:mm:ss' time! 'hh:mm:ss' money! just removes the '$ character string! 'sql-escaped-string' binary! 'sql-escaped-string' block! 'ENUM(string1, string2, ...)' All other REBOL types will be FORM-ed !
sql-escaped-string are REBOL strings converted to SQL format. Here's a list of the characters escaped for SQL :
REBOL format SQL format Description ^@ \0 Null ^/ \n Line feed ^- \t Tabulation ^M \r Carriage return ^H \b Backspace ' \' Simple quote " \" Double quote \ \\ Backslash
You can change the default conversion rules applied to each MySQL type using the following globally-defined function :
wherechange-type-handler port type handler
port is an opened port to a MySQL server. type is a valid MySQL type name. (See this table for allowed names) handler is a block! containing the conversion code. The handler works like a pipe :
orconverted value <= [handler] <= raw value (string!)
So you have to make a correct "piping" REBOL code, which takes a value from the right side and returns the converted value on the left side. You could make your own conversions after retrieving records with COPY, but using CHANGE-TYPE-HANDLER will ensure you the best performances, because your conversion code will be executed at low-level in a special 1-pass conversion process. This is especially true if you're retrieving a lot of records.new-value: handler raw-valueExample:
Convert 'short values to REBOL hex strings :
Convert 'blob to REBOL objects :change-type-handler db 'short [to-hex to-integer]
Assuming that your objects have been stored in the following way :to-obj: func [value][first reduce load to string! decompress] change-type-handler db 'blob [to-obj]
Note:to-blob: func [obj [object!]][compress mold obj] insert db ["INSERT INTO table1 VALUES (?)" to-blob my-obj]
- You can suspend at any time the conversion engine by setting the port/locals/auto-conv? property to OFF. This will speed up the driver a lot but all values will be string! values.
- You can check at any time the type conversion rules with:
>> probe your-port-name/locals/conv-list
- The type handlers are specific to a connection. So if you install your own handlers for a connection , they won't affect any previous or new connection handler!
If you feel comfortable with MySQL, give a try to the driver and report me all bugs and your comments about it! If you need a better support for some features (or features i didn't implement), feel free to send me all your needs/ideas.(If you find some bugs in this driver, you can add to your report a copy of the console output with trace/net activated.)
I would like to send a special thank to :
for helping me debug and improve this driver. (Sorry if I missed someone)Moulay Semlali Terry Brownel Gabriele Santilli Allen Kamp Frank Sievertsen Michal Kracik Petr Krezenlok Will Arp
- Improve the command-line client. (add multiline typing support)
- Make a graphical client and admin tool.
Copyright notice:
If you use the MySQL driver in a product, I would appreciate *not* receiving lengthy legal documents to sign. The sources are provided for free but without warranty of any kind. The driver has been entirely written by Nenad Rakocevic; it does not include third-party code.(C) 2001-2008 SOFTINNOV This software is provided 'as-is', without any express or implied warranty. In no event will the author be held liable for any damages arising from the use of this software. Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions: 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 3. This notice may not be removed or altered from any source distribution. Nenad Rakocevic [email protected]If you redistribute modified sources, I would appreciate that you include in the mysql-protocol.r file history information documenting your changes.