Interactive COBOL Web Message Utility Revision 3.55 17-Feb-2006 The ICOBOL WEB Message utility (ICWEBMSG) facilitates the sending of an HTTP or HTTPS request and receiving the response via a command-line utility. The intention is that ICOBOL programs can use the CALL facility to execute this utility to implement various forms of web communication. The default is to send an XML request file and receive a response. ICWEBMSG acts as a very simple client. ICWEBMSG is currently available for Windows, Linux2, Linux4, and SCO OpenServer. Under Windows, it is part of the Full installation package and can be installed as an additional optional component. It is not installed by default. ICWEBMSG is a stand-alone utility. It does not require the shared area allocated by ICEXEC. Under UNIX, it does not use any ICOBOL shared libraries, so it can be moved back to any previous version of ICOBOL. (Just remember to move the help file icwebmsg.hf, also). Under Windows, it uses the icsys.dll and two additional .dll's, ssleay32.dll and libeay32.dll. 3.55 ---- 1. Force close SSL connections to keep them from being cached. 3.54 ---- 1. ICWEBMSG will continue reading the data stream when the Response Line shows an error and the Content-Length is positive. A non-zero exit code (usually 1) will still be returned from ICWEBMSG. The response file should be viewed for error information. 2. Updated verbose logging messages. See change bars in logging section below. 3. When using SSL, sometimes data was not being received. 3.53 ---- None. 3.52 ---- None. 3.51 ---- None. 3.50 ---- 1. Added two new startup options, -C sec and -R sec added: -C sec (Connection wait time) The Connect-wait switch (-C sec) specifies the time, in seconds, that the ICWEBMSG utility will wait while trying to connect to the target-url. If the connect is not complete within that timeframe then a "Device Timeout" error will be returned and the utility will terminate with an exit code of 1. If not specified, the default is to wait forever on the connection. -R sec (initial Response wait time) The Response-wait switch (-R sec) specifies the time, in seconds, that the ICWEBMSG utility will wait for the initial response from the target-url. If no response is received within that timeframe then a "Device Timeout" error will be returned and the utility will terminate with an exit code of 1. If not specified, the default is to wait forever on the response. 2. Error handling has been improved. Status codes of 300 and up will have the response read and placed into the log file (for GET and PUT) and into the output file and log file for HEAD. For example, Status 302 a Temporary redirect can be detected and the returned Location: will hold the new URL to process. 3. The output-file is now optional. If not given, it defaults to stdout. Under UNIX, this is useful by using UNIX Pipe Opens (I.e,, OPENs with "|<" or "|>" as the first characters of the filename). 3.46.01 ------- 1. Fixed a case where the output file was being dumped improperly. Documentation ------------- Syntax ------ The syntax for icwebmsg is: icwebmsg [options]... target-url [output-file] -a[:aflag] Audit to %P.lg -A path[:aflag] Audit to path, or path%/%P.lg if path is a directory flag is a|b|d|da|db|p|pa|pb|t|ta|tb|u|ua|ub a=append, b=backup, d=datestamp, p=process-id, t=timestamp, u=username -C sec Connection wait timeout in seconds (default is to wait forever) -F value emit From: header with "value" (default: no From: header) -h | -? display Help text -I path Include content of "path", read and send it (POST only, required) -L value emit content-Language: header with "value" (default: en-US) -M GET|POST|HEAD request Mode for HTTP (default is POST) -q Quiet operation -R sec Response wait timeout in seconds (default is to wait forever) -s emit SOAPAction: header with no value -S value emit SOAPAction: header with "value" -T value emit content-Type: header with "value" (default: text/xml) -U value emit User-Agent: header value appended after %P/%r -v Verbose logging to the audit file -V 1.0|1.1 Version of HTTP to use (default is 1.0) Notes: target-url may only specify HTTP or HTTPS -I, -L, -s, -S, -T only apply to the POST method if output-file is not given, STDOUT is assumed. Environment variables: ICROOT Messages and help ICWEBMSG Command line options This utility takes the standard options (-a, -A, -h, -?, -q) for auditing, help, and quiet operation. Also, as is standard, an environment variable by the same name can be set with command line options that you want to use all the time (e.g., ICWEBMSG=-aq will always audit and always run quiet mode) The first argument is required and is the target URL. The second argument is the path to the file that will receive the result, if not specified, it defaults to STDOUT. The URL must start with either HTTP or HTTPS. The Secure Sockets Layer (SSL) services are provided by the openssl project. The copyrights are included later in this readme, and a snapshot of the sources used to build with are available upon request. Several of the parameters control the various HTTP headers that are sent with the request. If the structure of an HTTP request is not familiar, there is a simple and helpful HTTP tutorial at: http://www.jmarshall.com/easy/http/ The following options are supported to further control the web request: -C sec Connection wait timeout in seconds (default is forever) -F value emit From: header with "value" (default: no From: header) -I path Include content of "path", read and send it (POST only, required) -L value emit content-Language: header with "value" (default: en-US) -M GET|POST|HEAD request Mode for HTTP (default is POST) -R sec Response wait timeout in seconds (default is forever) -s emit SOAPAction: header with no value -S value emit SOAPAction: header with "value" -T value emit content-Type: header with "value" (default: text/xml) -U value User-Agent: header with "value" appended after %P/%r -v Verbose logging to the audit file -V 1.0|1.1 Version of HTTP to use (default is 1.0) Let's cover them one by one. -C seconds (CONNECT WAIT TIMEOUT) ---------- This parameter specifies the time in seconds that the utility will wait for a connection to the target-url. If no response is received within this timefrace a "Device Timeout" error will be given and the utility will terminate with an exit code of 1. Valid values are 0 to 6300. (I.E., 1 hr and 45 minutes). If not specified, ICWEBMSG will wait forever. -F value (FROM) -------- The From: header usually specifies the email address of the person or company responsible for sending the request. It is optional. This field is often transcribed into the web server logs. It may be helpful for debugging if the username or console number of the person making the request is supplied. This header is omitted by default. -I path (INCLUDE) ------- When a POST request is sent, the data that the web server is to process must be supplied. This parameter specifies the path to a file that will be sent. The file is sent, byte by byte to the web server as the "content". There is no translation of any sort. POST is the method used by XML servers, and the content is the xml file with the line as the first line. -L value (LANGUAGE) -------- The Language-Encoding header is supplied when POST is used and indicates to the web server the language being used by the content being sent. The value supplied here is simply passed on in the header line. en-US is supplied as the default. -M GET|POST|HEAD (METHOD) ---------------- This parameter selects the basic request method being used. The default value is POST. GET is the method used by a web browser when you type in a URL to get a page. The content that is returned is copied byte-by-byte into the output file. The returned headers are not copied. HEAD only returns status headers and no content. Those headers are copied to the output file instead of content. POST is the method that is often used by a browser to send the data that you filled out in a form. As previously mentioned, it is also often used by XML servers that communicate via HTTP. -R seconds (RESPONSE WAIT TIMEOUT) ---------- This parameter specifies the time in seconds that the utility will wait for an initial response to its request to the target-url. If no response is received within this timefrace a "Device Timeout" error will be given and the utility will terminate with an exit code of 1. Valid values are 0 to 6300. (I.E., 1 hr and 45 minutes). If not specified, ICWEBMSG will wait forever. -s (SOAPACTION) -S value -------- This parameter specifies that the SOAPAction header should be sent and the value to supply with that header. The -s option says to send the header, but the value is empty. The -S option passes whatever value specified. This header is required by some web/SOAP servers, but it is one that has been replaced by other fields in the SOAP content. If your system doesn't need it, don't set it. -T value (TYPE) -------- This parameter specifies the Content-Type header that is supplied when POST is used. The default value is text/xml. Supply whatever value is appropriate for the server and the type of request you are sending. It should be a valid MIME type or the web server may return an error. -U value (USERAGENT) -------- One of the headers that is always sent is the User-Agent header. This header usually specifies the name and revision of the program that is sending the request, and it is usually logged in the web server logs. It is always set to icwebmsg/3.xx. If this parameter is specified, a space will be appended followed by the value specified For example, if "-U CreditCheck/2.0" was specified, the header will look like: User-Agent: icwebmsg/3.50 CreditCheck/2.0 If this is set to the name and revision of the COBOL program that calls ICWEBMSG, it will help as you debug your system. -v (verbose) -- This parameter indicates that ICWEBMSG is to perform verbose logging to the audit log. When this parameter is NOT set, only the data that would come out on the screen is sent to the audit log. When specified, the following additional information is logged: A copy of the headers that are sent Progress and size of any content that is sent Headers that are received as part of the response Progress and size of any content that is received Any trailers (possible with chunked data in HTTP/1.1) received An item to note is the "Finished ... lines" provided by icwebmsg that have a timestamp (HH:mm:ss.hh) at the end of each line. This timestamp can provide timing information in case the Connect-wait and/or Response-wait switches need to be given, or just to see the portions of time in the round-trip for each section. General Format for verbose switch logging: Secure: No/Yes Host: xx Port: n URL: xx Input: xx (for POST) Output: xx SSL Connection informatiom... (for Secure Sockets) Finished connecting: HH:mm:ss.hh | Writing Headers... method url HTTP/version Host: xx Connection: xx From: xx (optional) User-Agent: xx SOAPAction: xx (optional) Content-Language: xx (for POST) Content-Type: xx (for POST) Content-Length: n (for POST) | Finished writing headers: HH:mm:ss.hh | Writing bytes of content... (for POST) | Finished writing content: HH:mm:ss.hh (for POST) | Reading Response... | ...(actual response) Finished reading response: HH:mm:ss.hh | Reading header... | ...(actual header) Finished reading headers: HH:mm:ss.hh | Reading bytes of content (for GET and POST) or Reading content in chunks (this section for chunk response) Reading chunk of bytes . End of chunks . | Content size was bytes . | Reading trailer... (optional) | ...(actual trailer line(s)) (optional) or Reading content of unknown length | Content size was bytes Finished reading content: HH:mm:ss.hh -V 1.0|1.1 (VERSION) ---------- This parameter specifies the HTTP version that is set in the headers that are sent with the request be set. The default is 1.0, which should be recognized by virtually any web server. If version 1.1 is set, icwebmsg sends some additonal headers needed (or desired) by the 1.1 protocol. The simple tutorial referenced above explains some of these. Error Handling -------------- Errors will be reported in the usual way - via stderr and into the audit file if one is specified - and they will result in a nonzero exit code. The exit code values follow the same conventions as other ICOBOL utilities. We strongly suggest using an audit log - perhaps with a filename based on console number, and using the append option. That way you can see the entire set of transactions for a particular console. While you're getting the application up and running, use the -v switch to see everything going back and forth. It will let you track down most problems very quickly. Besides errors from processing the command line, there can be errors for a failure to make a connection, i/o errors on the connection once it is made, filename processing errors, file open/create and i/o errors writing the data coming back, malformed responses or response headers, etc. The other errors that can come back are HTTP errors, like 404 NOT Found. The utility inspects this value. Any value 300 or greater will be treated as an error. It will be reported and processing will stop after the header is read. If there is an error and if nothing has been written yet to the output file, the output file will be deleted. If, however, some data was written, it is not deleted. Sometimes inspecting the partial result will help track down failures on the web-server side. Some Web sites that may be helful while debugging are: http://web-sniffer.net (provides an http viewer) http://www.rexswain.com/httpview.html (another http viewer) You can also manually experiment with HTTP by using telnet. Using telnet, open an interactive socket to an HTTP server. Then manually enter a request, and see the response written to your screen. It's a great help when learning HTTP, to see exactly how a server responds to a particular request. It also helps when troubleshooting. For example, to open a connection to an HTTP server would be something like: telnet www.somehost.com 80 Then enter your request line by line, like GET / HTTP/1.0 [headers here, if any] blank line here (Remember you may have to send to end each line (Ctrl-M, Ctrl-J) and you may also want to enable echoing in the telnet to see what you are typing. Also the blank line is REQUIRED to end the request.) After finishing your request with the blank line, you'll see the raw response from the server, including the status line, headers, and message body. EXAMPLES -------- Note that all the samples use the -a and -v switches to specify auditing and verbosity. Example 1 - GET --------------- One very simple example is using the GET method to read a standard web page. To get the default icobol web page you could use: icwebmsg -av -M GET http://www.icobol.com icobol.in.htm This will get the default web page from the icobol.com site and store it in the file icobol.in.htm Example 2 - POST ---------------- Another example using the POST method would be: An example input file (called request.xml) might be: (from the OpenSOAP project) foo The response (response.htm) received back looks like: Hello, foo! The command line to implement this used the defaults and would be something like this: icwebmsg -av -I request.xml http:///cgi-bin/HelloService.cgi response.htm From the OpenSoap project. www.opensoap.jp Example 3 - POST ---------------- Another example using the POST method that gets the current time from www.soapware.org An example input file (called gettime.xml) would be: (from www.soapware.org) The response (gettime_res.htm) received back looks like: 2004-07-15T07:01:38-04:00 The command line to implement this used the defaults and would be something like this: icwebmsg -av -I gettime.xml -S \"/currentTime\" http://time.soapware.org gettime_res.htm NOTE: The quote characters in the -S argument must be escaped such that the shell or command processor does not remove them as the entire string "/currentTime" (including the quotes) is required for the SOAPAction. Another use is to perform a pipe-open in ICOBOL and read the input. For example using the logon program, select U for utilities, and then L for list. Enter "|