Notes - Gateway/400 Group

Transcription

Generating XML
With RPG
Jon Paris
Jon.Paris @ Partner400.com
www.Partner400.com
www.SystemiDeveloper.com
Notes
This presentation may contain small code examples that are furnished as simple examples to provide an illustration.
These examples have not been thoroughly tested under all conditions. We therefore, cannot guarantee or imply
reliability, serviceability, or function of these programs.
All code examples contained herein are provided to you "as is". THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED.
The author, Jon Paris, is co-founder of Partner400, a firm specializing in customized education and mentoring
services for IBM i developers. With his partner, Susan Gantner, Jon authors regular technical articles for a number of
technical publications including IBM Systems Magazine, and its companion electronic newsletter, Extra and their
weekly iDevelop blog (ibmsystemsmag.blogs.com/idevelop). You may view articles in current and past issues and/or
subscribe to the free newsletter or the magazine at: www.ibmsystemsmag.com
Susan and Jon are also partners in SystemiDeveloper, a company that hosts the RPG & DB2 Summit conferences.
See SystemiDeveloper.com for more details.
Feel free to contact the author at: jon.paris @ partner400.com
or visit the Partner400 web site at www.partner400.com
© Partner400, 2015
Gateway/400 Seminar - December 10th, 2015
Lecture Notes: Page 1 of 146
Agenda
What is XML - A Brief Refresher
• XML's Relationship to HTML and DDS
• Writing XML: The Basics
✦
Terminology, syntax and basic rules
Creating XML
• Doing it the hard way
• Templating to the rescue
✦
Using CGIDEV2
• Larry Ducie's XMLi
✦
xmli.sourceforge.net
• Henrik Rützou's powerEXT Core
✦
www.powerext.com/pextdoc_CGI.htm
There are also now improved options for generation and consumption
of XML with DB2 and SQL features
• We will look very briefly at the XML generation capabilities later
Notes
© Partner400, 2015
What is XML?
eXtensible Markup Language
• A markup language for creating other markup languages!
Markup languages use tags inserted into the actual information
• Examples include HTML (Web pages), UIM (IBM i Menus etc.)
The user defines a "language" to format specific information
• i.e. "You" can create/define your own tags
Defines both the content and nature of the information
• Unlike HTML which just defines how to display data in a browser
Major focus is to standardize information representation
• Thereby enhancing information interchange
XML is the syntactic foundation for many technologies
• Many web services - including SOAP based services
• Microsoft Office files ( Word, Excel, ... )
• Even this presentation
"You" means your company or industry body or government or ...
Notes
© Partner400, 2015
HTML Comparison Example
<html>
<br>
<CENTER>
<table Border=1 CellSpacing=1 CellPadding=5 Width="600">
<th>Contact
<th>Company
<th>Address
<th>City
<tr>
<td>Sally Smith</td>
<td>Acme Computer Supplies</td>
<td>1234 Jones St.</td>
<td>Anytown, MN 55901</td>
</tr>
<tr>
<td>John Jones</td>
<td>Phones R Us</td>
<td>5678 High Street</td>
<td>Mytown, GA 30033</td>
</tr>
<tr>
<td>Meanold Miser</td>
<td>Suchadeal Bank</td>
<td>91011 Important Ave.</td>
<td>Jonesville, MN 55901</td>
What if we needed to
search this document for all
people named "Jones" ?
Notes
In this example you can see that HTML is concerned only with how the data is laid out on the web page. It is difficult
to derive from this the nature of the information and as a result it is difficult for computers to associate meaning with
the data. The result is that searching contextually can be very difficult.
© Partner400, 2015
HTML Example, continued
The HTML would look like this in the browser
• Pretty enough, but the data has no context
✦
Only what we associate with it through our personal experience
• HTML tags can only give us a presentation "view"
✦
✦
In other words they only control the appearance
They tell us nothing about the content
Basic XML Version
<?xml version='1.0'?>
<Customers>
<Customer>
Pretty ...
<Contact>Sally Smith</Contact>
<Company>Acme Computer Supplies</Company>
<Address>
<Street>1234 Jones St.</Street>
<City>Anytown</City>
<State>MN</State>
<Zip>55901</Zip>
</Address>
</Customer>
Or pretty
<Customer>
<Contact>John Jones</Contact>
<Company>Phones R Us</Company>
<Address> <Street>5678 High Street</Street>
<City>Mytown</City>
<State>GA</State>
<Zip>30033</Zip> </Address> </Customer>
<Customer> <Contact>Meanold Miser</Contact>
<Company>Suchadeal Bank</Company>
© Partner400, 2015
messy
The meaning is
the same with
XML
XML Example, continued
Note the descriptive names for the elements
• Emphasis is now on data content, not presentation
✦
Now we know what this data represents!
• Think how easily you can search for a contact name of "Jones"
✦
Also much easier to share information with other applications or systems
• An XML document is roughly analogous to the data in a physical file
✦
But with the field names embedded
<Customer>
<Contact>Sally Smith</Contact>
<Company>Acme Computer Supplies</Company>
<Address>
<City>Anytown</City>
<State>MN</State>
<Zip>55901</Zip>
</Address>
</Customer>
<Customer>
<Company>Phones R Us</Company> ...
XML Terms and Syntax
Elements
• A combination of a tag and its corresponding data
• Elements may contain other elements
✦
The required Root element contains all other elements in the document
• XML documents consist of elements, such as:
Opening tag Content
Closing tag
• Complex elements may contain other elements
✦
Similar to a Data Structures in RPG
Attributes
• Additional information about an element
<Address Type="Shipping">
Attribute
Note that all element and attribute names in XML are CaSe SeNsItIvE!
© Partner400, 2015
What is a Schema?
The definition of the customized markup language
• It defines the tags and the rules for using them
✦
e.g. Element names, sequence, attributes: optional and required
• Used to validate an XML document
✦
i.e. Does the document use the data elements correctly?
• Schemas are roughly analogous to the DDS for a PF
✦
Except arrays and data structures can be defined as well as simple fields
There are 2 ways to write schemas:
• DTD: Document Type Definition
✦
✦
Most widely used method to date
Ugly syntax, not consistent with XML syntax and very limited in function
-
Will hopefully be replaced over time with XML schemas
• XML Schema
✦
✦
Richer, more robust syntax consistent with XML documents
Developed as a standard by W3C
One small problem
• There is currently no standard validator on IBM i
• Although you can now use SQL for the purpose
Notes
© Partner400, 2015
One More Thing - Namespaces
XML allows you to define your own "language"
• So there is always the possibility of a conflict in element names if you blend
documents from multiple sources
Namespaces allow you to deal with this by qualifying the element names
• Similar to Qualified DS in RPG
This example below is based on one from W3 Schools
• There is a useful free tutorial on XML on their web site
<namespace>
Here <table> is used to
<h:table>
identify a list of items
<h:tr>
<h:td>Apples</h:td>
<h:td>Bananas</h:td>
</h:tr>
</h:table>
But here it relates
to furniture!
<f:table>
<f:name>African Coffee Table</f:name>
<f:width>80</f:width>
<f:length>120</f:length>
</f:table>
</namespace>
Notes
Namespaces are not really an issue when generating XML documents, that comes into play more when processing
them. However, you need to understand what they are and how they fit into the picture.
© Partner400, 2015
Creating XML Documents With RPG
The Hard Way
and the Easier ways
Notes
I am always surprised by the number of people who build XML documents the "hard way". It is fine to create a simple
XML string for a web service call this way, but to create documents for transmission to business partners,
government, etc. it is just too much work for me. It is bad enough the first time you do it, but maintenance can be a
nightmare - and the requirements for the document _will_ change.
I will show you three different tools, all of them free, that you can use to simplify the task.
© Partner400, 2015
Creating XML The Hard Way
Concatenate literals/constants with field content
• Store in varying length field(s)
Use IFS APIs to write data to stream file
• Not familiar with those APIs?
✦
Write strings to database and then use CPYTOIMPF to create IFS files
• Or use Scott Klement's tutorial to learn how to use the APIs
This approach is tedious, error-prone, and just plain ugly!
• Not to mention that it is a maintenance nightmare
Dcl-s XMLdata VarChar(1000000);
Dcl-s len
INt(10);
...
Dou customersProcessed;
XMLdata += '<Customer><CustomerNo>' + %EditC(CUSTNO: 'X') +
'</CustomerNo><Address><StreetDetail>' + %TrimR(STREET) +
...
'</Address></Customer>';
EndDo;
len = write(fHandle: ( %Addr(XMLdata: *Data) ): %Len(XMLdata));
Notes
The coding used in the chart makes use of a couple of V6 features.
✦
✦
The ability to handle variables longer than 64K - here we have defined a variable of 1,000,000 bytes.
The ability to specify to %Addr that we want the address of the data portion of the variable length field, not the
start of the field itself as that would include the field length definition area (the first four bytes in this case).
If you are on an older system then the code would have to be modified to:
1) Reduce the size of the field to 65,535 or less and
2) Modify the code %Addr(XMLdata: *Data) to %Addr(XMLdata) + 2 .
Of course you would also probably have to allow for the fact that you might want to issue a write() for every record
processed as 64K is not a lot of data and is probably not large enough for many applications.
© Partner400, 2015
Creating XML - The Easy Way
Use a templating system
• This example uses the one supplied by CGIDEV2
• It was originally designed to produce web pages
✦
But as we've noted XML and HTML are very similar
It is far more flexible
• The XML template is external to the program
✦
Just like DDS externalizes the details of a display file
• Change the template and you have changed the XML
✦
Without even having to recompile
The complexity of the process is masked by the CGIDEV2 APIs
GethtmlIFS('/www/xmltemplates/Customers.xml';
UpdHTMLVar('CustNo' : %EditC(CUSTNO: 'X') );
UpdHTMLVar('Street' : STREET );
UpdHTMLVar('City'
: CITY );
We will be discussing the
WrtSection('Customer');
details behind this code on
subsequent charts
CGIDEV2
What is it?
• A set of RPG IV subprocedures developed to simplify web programming
• IBM supplied it free of charge and all source code is included
✦
But the best and most up-to-date distribution is from www.easy400.net
How do we use it?
• You supply a template that defines the "shape" of the XML document
• It includes special markers to indicate the "Sections"
✦
i.e. Record format names
• Plus markers to indicate where variable data is to be placed
✦
Operates much the same as DDS as we'll see on the next chart
• See this article for a complete walk-though of the process
✦
www.ibmsystemsmag.com/ibmi/developer/rpg/Using-CGIDEV2-for-Generating-XML/
/$Customer
<Customer>
<CustomerNo>/%CustNo%/</CustomerNo>
<StreetDetail>/%Street%/</StreetDetail>
<CityName>/%City%/</CityName>
</Customer>
© Partner400, 2015
Comparing an XML Template to DDS
A
A
A
A
A
A
A
R CUSTOMER
CUSTNO
/$Customer
6S 0
STREET
30
CITY
25
5
5
7
7
9
9
11'Customer No'
31
11'Street Detail'
31
11'City Name'
31
Substitution markers
CustNo, Street and City
are replaced at run time by
the actual field values - just
as they would be in a
display or printer file
Literals such as
'<CityName>' are output
as-is just as they would be
in DDS.
<Customer>
<CustomerNo>/%CustNo%/</CustomerNo>
<StreetDetail>/%Street%/</StreetDetail>
<CityName>/%City%/</CityName>
</Customer>
Defining an XML Template
This example use the default section identifier "/$"
Notice that there are three sections (formats)
• The first (Header) will be output at the start of the file
• The second (Customer) is used for the repeating customer elements
• The third (Trailer) wraps up the document
The whole template can be placed in a source physical file or in an IFS file
/$Header
<?xml version='1.0'?>
<Customers>
/$Customer
<Customer>
<Contact>/%Contact%/</Contact>
<Company>/%Company%/</Company>
<Address>
<Street>/%Street%/</Street>
<City>/%City%/</City>
<State>/%State%/</State>
<Zip>/%Zip%/</Zip>
</Address>
</Customer>
/$Trailer
</Customers>
© Partner400, 2015
The RPG Code to Process the Template
GetHTML loads the template into memory
UpdHTMLVar supplies the data for the substitution variables
✦
Note that %Char (or %EditC/W) must be used for all numeric fields
WrtSection actually merges the skeleton with the variable content
✦
A substitution variable can appear multiple times just as a DDS variable can
WrtHtmlToStmf writes the buffered XML to the IFS
✦
The second parameter identifies the code page of the file to be created
// Load HTML from source file and write out the Header Section
gethtml('HTMLSRC': '*LIBL': 'CUSTXML');
WrtSection('Header')
// Update substitution variables for each record & write it out
UpdHTMLVar('Contact': ContactName);
UpdHTMLVar('Company': CoName);
This portion is
UpdHTMLVar('Street' : Address1);
repeated for each
UpdHTMLVar('City'
: City);
UpdHTMLVar('State' : StateCode);
record to be output
UpdHTMLVar('Zip'
: %EditC(ZipCode: 'X'));
WrtSection('Customer');
// When all records processed write the trailer and output to the IFS
WrtSection('Trailer');
WrtHtmlToStmf('/xml/Customers.xml':819);
Alternatives to CGIDEV2
Henrik Rützou's PowerEXT (powerext.com)
• Developed as part of the PowerEXT web development framework
• API driven approach that can handle HTML, JSON, XML and more
✦
Because it is also based on CGIDEV2 you can also use a template approach
• XML facilities are part of the free Core features
✦
www.powerext.com/pextdoc_CGI.htm
Larry Ducie's XMLi Toolkit (xmli.sourceforge.net)
• Designed for XML from the ground up
• Two "flavors" of operation
✦
✦
API driven
Template driven
• See www.ibmsystemsmag.com/ibmi/developer/rpg/xmli/ for worked
examples
© Partner400, 2015
powerEXT
An API based approach
• powerExt lets you create an element and add data in a single call
✦
Element attributes can be specified in a similar manner
• XMLi is similar but requires you to specify the name of the element being closed.
Dcl-Ds customer ExtName('QCUSTCDT') End-Ds;
// Declare Internal Variables
Dcl-S recordCount Int(5);
Dcl-C endOfData
'02000';
Dcl-S xmlFilename Varchar(128) Inz('/Customers.xml');
clearSrvPgm();
setContent('*none');
Exec SQL
select count(*)
into :recordCount
from qcustcdt;
xmlNode('Customers');
xmlNode('RecordCount': '': %char(recordCount));
Notes
powerExt (www.powerExt.com)
The powerExt approach is to use individual API calls to add nodes and attributes to the XML tree. As you can see in
the example, the same fundamental API call is used to add a node regardless of whether it is an elementary item or
a group item. The only difference is whether the element value itself is present. The API XMLendnode() is used to
close off open elements. There is no need to specify the name of the element which is ending as the rules fro
formulating XML make it possible for the tool to work out for itself the element name to use.
Fixed-form D-specs are below in case you haven't fully come to grips with free form definitions yet:
d customer
E DS
ExtName(QCUSTCDT)
// Declare Internal Variables
d recordCount
s
5i 0
d endOfData
c
'02000'
d xmlFilename
s
128a
Varying Inz('/Customers.xml')
© Partner400, 2015
powerExt - continued
Exec SQL
declare customerCursor cursor for
select * from QIWS.QCUSTCDT;
Exec SQL
open customerCursor;
DoU SQLSTATE = endOfData;
Exec SQL
fetch next from customerCursor into :customer;
If SQLSTATE <> endOfData;
xmlNode('Customer':'ID="'+ %char(cusnum) + '"');
xmlNode('Name':'':LSTNAM);
Starts
xmlNode('Address');
xmlNode('Street':'':street);
element and
xmlNode('City':'':city);
adds attribute
xmlNode('State':'':state);
value
xmlNode('Zip':'':%editc(zipcod:'X'));
xmlEndNode();
xmlEndNode();
Note: No element
EndIf;
name needed
EndDo;
Exec SQL
close customerCursor;
xmlEndNode();
echoToStmf(xmlFilename:1208);
Notes
© Partner400, 2015
XMLi Toolkit - The Template Approach
The template text is in a file in the IFS (Template 1.xml)
• It uses an XSL transformation style i.e. for-each ...
<xmli:template xmlns:xmli="http://www.sourceforge.net/xmli" ccsid="1208">
<Customers>
<xmli:sql name="custCount" statement="select count(*) from mcustomers">
<xmli:for-each>
<RecordCount><xmli:value-of select="custCount.1" /></RecordCount>
</xmli:for-each>
<xmli:sql name="custRow" statement="select * from mcustomers">
<xmli:for-each>
<Customer ID="${custRow.1}">
<Company><xmli:value-of select="custRow.2" /></Company>
<Address>
<Street><xmli:value-of select="custRow.3" /></Street>
<City><xmli:value-of select="custRow.4" /></City>
<State><xmli:value-of select="custRow.5" /></State>
<Zip><xmli:value-of select="custRow.6" /></Zip>
</Address>
</Customer>
</xmli:for-each>
</Customers>
<xmli:write-to-file path="'/Partner400/Redbook/Customers.xml'" />
</xmli:template>
Notes
The template option offered by XMLi appears at 1st glance to be similar to the template approach of CGIDEV2, and
in some respects that is a valid comparison. However XMLi templates go far beyond the capabilities offered by
CGIDEV2.
XMLi templates can incorporate SQL so that the entire data retrieval and XML build process can be significantly
simplified.
In this simple example you will see that some aspects of the template look remarkably similar to the CGIDEV2
version. However the code to populate the template is very different, but hopefully mostly self-explanatory. The most
significant difference being the way in which the values are set for the parameters.
Note that when extracting data for use as the value of an element we use the value-of construct. However, we have
to use a slightly different approach when we need to embed a value in the middle of an XML tag. You can see an
example of this above with the Customer ID attribute. You code the ${ } sequence to wrap any code that XMLi will
interpret and replace. In this case:
<Customer ID="${custRow.1}">
We’re just using it to retrieve the first value from the result set.
© Partner400, 2015
XMLi Toolkit - The Template Approach
This is all the RPG code needed to run the template
• There is a command option which can avoid the need for any RPG logic
All selection / processing logic is contained within the template
• Selection criteria etc. can be also passed from RPG as parameters
/include XMLI_H
Dcl-S templateFilename Varchar(128)
Inz('/Partner400/Redbook/Template 1.xml');
// Load and run the template...
xmli:loadTemplate('CUST'; : templateFilename);
xmli:runTemplate('CUST');
// Unload template (Could be left loaded for further use)...
xmli:unloadTemplate('CUST');
*InLR = *On;
Notes
As you can see very little RPG is needed to initiate and run the template. So little in fact that in simple cases such as
this the program could have been replaced by a simple command that comes with the package.
Fixed-form D-specs:
/include XMLI_H
D templateFilename...
D
s
D
D
© Partner400, 2015
128a
Varying
Inz('/Partner400/Redbook+
/Template 1.xml')
XMLi - API Method
XMLi's API approach is very similar to that of powerExt
• xmli_addElement() adds a simple element with value
• xmli_openTag() starts a complex element
✦
Attributes can be added with xmli_addAttribute()
• xmli_closeTag() is used to close elements by name
...
If SQLSTATE <> endOfData;
xmli_openTag('Customer');
xmli_addAttribute('ID': %char(cusnum));
xmli_addElement('Name':'': LSTNAM);
xmli_openTag('Address');
xmli_addElement('Street': street);
xmli_addElement('City': city);
xmli_addElement('State': state);
xmli_addElement('Zip': %editc(zipcod:'X'));
xmli_closeTag('Address');
xmli_closeTag('Customer');
EndIf;
EndDo;
Exec SQL
close customerCursor;
...
Notes
The first of XMLi's modes of operation is an API approach which as you will see later is very similar to that used by
powerExt. The principal difference is that with XMLi when closing an element it is necessary to specify the name.
© Partner400, 2015
Creating XML with DB2
Quite easy for relatively straightforward tasks such as our examples
• But complexity level accelerates very rapidly when multiple tables and
selective output is required
✦
Personally find the syntax confusing and completely non-intuitive
My take - It really needs a tool to build the SQL statements
Dcl-s MyXMLDoc
Exec SQL
SQLTYPE(XML_CLOB_File);
Set Option
Commit=*None, DatFmt=*ISO, CloSQLCsr=*EndActGrp;
MyXMLDoc_Name = '/Partner400/XMLStuff/WrtSQLXML3.xml';
MyXMLDoc_NL
= %Len(%Trim(MyXMLDoc_Name));
MyXMLDoc_FO
= SQFOVR;
//Replace if file exists
Exec SQL
<?xml version="1.0" encoding="UTF-8"?>
Select XmlDocument
<Customers>
<Customer>
(xmlgroup(CustNo as "ID",
<ID>A0011</ID>
(Trim)Name as "Cust-Name",
<Cust-Name>Acme Best Brew</Cust-Name>
(Trim)City as "City",
<City>Acme</City>
<ZipCode>0</ZipCode>
Zip as "ZipCode"
</Customer>
Order By City, Name
<Customer>
Option Row "Customer"
<ID>J0003</ID>
Root "Customers"))
<Cust-Name>JB's Brew Pub</Cust-Name>
<City>Anaheim</City>
into :MyXMLDoc
....
From Customers;
Notes
The generation, and for that matter consumption, of XML using SQL has come a long way over the years but I personally still find the syntax overly complex and confusing. In a simple example such as
the one shown above it doesn't matter that much. But for more complex documents, the degree of difficulty seems to increase rapidly and soon approaches what I would consider an example of "write
only" logic - i.e. you may be able to write it and get it running, but can you subsequently read and modify it? Here's a more complex example to show you what I mean:
EXEC SQL
With Address as (Select CustNo, XMLElement(Name "Customer",
XMLAttributes(Trim(CustNo) as "CustNo"),
XMLConcat(XMLElement(Name "Name",
Trim(Trim(CustName1) concat ' ' concat Trim(CustName2))),
XMLForest(Trim(Street) as "Street",
Trim(ZipCode) as "ZipCode",
Trim(City) as "City"))) CustXML
From Addressx),
Header as (Select Company, OrderNo, CustNo,
XMLForest(Company as "Mandant",
OrderNo as "OrderNo") OrderNoXML,
XMLConcat(XMLElement(Name "DeliveryDate", Char(DelDate, ISO)),
XMLElement(Name "OrderType",
Case When OrderType = 'DO' Then 'Domestic'
When OrderType = 'EX' Then 'Export'
When OrderType = 'UO' Then 'Express'
Else '???' End),
XMLElement(Name "DeliveryTerms",
Case When DelTerms = 'CPT' Then 'Delivered Free'
When DelTerms = 'EXW' Then 'Ex Works'
Else '???' End))
HdrAddXML
From OrderHdrX
Where DelDate between '2009-12-01' and '2009-12-31'),
HdrAddr as (Select Company, OrderNo,
XMLConcat(OrderNoXML, CustXML, HdrAddXML) OrderXML
from Header Join Address Using(CustNo)),
.....
And on it goes. This is just the first half or so of the statement.
Note: The actual output of the statement on the chart is not formatted as I have shown it - it comes out as one huge long line. I used the xml tooling in RDi to format it.
Many thanks to Birgitta Hauser for these examples - without her help I wouldn't have known where to begin.
© Partner400, 2015
Useful Reading, Tutorials, etc.
XML for the World Wide Web - Elizabeth Castro
• Low priced, easy to read, general purpose introduction.
✦
Well supported by website with downloadable examples
• Peachpit Press - www.peachpit.com (ISBN 0 - 201 - 71098 - 6)
XSLT - Doug Tidwell
• Not a light read but a comprehensive and entertaining treatment by one of
IBM's top XML "apostles"
• O'Reilly - www.oreilly.com (ISBN 0 - 596 - 00053 - 7)
W3Schools.com - "the best things in life ARE free"
• Tutorials on all aspects of XML, XSL, and more
• www.w3schools.com
ZVON.org - "The Guide to the XML Galaxy"
• Tutorials, links and more www.zvon.org
And still more information . . .
IBM alphaWorks
• Latest tools and enablers supporting XML
• alphaWorks.ibm.com
W3C - XML standards and specifications
• Status and detail on various XML-related standards
• www.w3.org/XML
XML.org - information on XML standards, tools
• Repository for industry standard DTDs shared across companies
• www.xml.org
© Partner400, 2015
Any Questions?
?
© Partner400, 2015
© Partner400, 2015
Processing XML
with RPG
Jon Paris
www.Partner400.com
Notes
This presentation may contain small code examples that are furnished as simple examples to provide an illustration.
These examples have not been thoroughly tested under all conditions. We therefore, cannot guarantee or imply
reliability, serviceability, or function of these programs.
All code examples contained herein are provided to you "as is". THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED.
The authors, Jon Paris and Susan Gantner, are co-founders of Partner400, a firm specializing in customized
education and mentoring services for AS/400 and iSeries developers. Together with Paul Tuohy, they also operate
the System i Developer education consortium which runs the RPG & DB2 Summit conferences. See (http://
systemideveloper.com/Summit/conferences.html).
Their individual careers include a number of years with IBM, including working at both the Rochester and Toronto
laboratories. These days Jon and Susan are devoted to educating developers on techniques and technologies to
extend and modernize their applications and development environments.
Jon and Susan author regular technical articles for the IBM publication, IBM Systems Magazine, i5 edition, and the
companion electronic newsletter, iSeries Extra. You may view articles in current and past issues and/or subscribe to
the free newsletter at: http://IBMsystemsMag/IBMi. They also write a weekly blog which you can find at: http://
ibmsystemsmag.blogs.com
Feel free to contact the authors: contact @ partner400.com
or visit the Partner400 web site at http://www.partner400.com
© Partner400, 2015
Agenda
The Basics
The XML processing opcodes and the %XML BIF
A simple XML-INTO example
Processing a simple document in one step
A more realistic XML-INTO example
Defining the Nested DS is the hardest part!
Post V5R4 updates to XML support
Namespaces, identifying optional elements, special characters
and more
Processing XML in "pieces" using %HANDLER
And finally a (very) brief look at XML-SAX
Notes
This is primarily a journey through the use of the XML op-codes in RPG, but along the way we’ll be touching on
some of V5’s support for nested Data Structures (a feature you must understand before you can fully utilize the XML
support).
We will also talk briefly about call-back processing - a programming technique that is common in other languages but
unfamiliar to most RPGers. Why do you need to know about it? Because the technique is utilized by the XML-INTO
op-code when you wish to handle large documents and is also a requirement when using XML-SAX.
We will not have time to cover XML-SAX in any detail, but have included some charts that will give you a “flavor” of
what it is all about.
© Partner400, 2015
The XML Op-codes
XML-INTO
• The more "RPG Like" of the two opcodes
• Processes the document as a whole
• Uses Data Structures to contain the extracted data
✦
Defining the DS can be the hardest part of using it
• Best when you need to extract all of the data in the document
XML-SAX
• Processes the document one piece at a time
✦
e.g. Start of element, element data, End of element, etc.
• Typically requires a lot more programming
• Best when you only need to extract specific elements
✦
Or when size restrictions come into play
Notes
Although there are only two opcodes involved, as you will see they do a lot of work.
XML-INTO is the powerhouse - the most challenging part of using it is in working out the DSs that need to be defined
to hold the extracted data. Once you have done that the rest is very simple.
XML-SAX is not as commonly used and it is difficult to come up with useful examples as every usage will be
different. We have included a few charts at the end to give you an idea of how it is used. But we may not have time
to get to them as there is a lot of material to cover. If you have specific questions about XML-SAX and we are not
able to get to them during the session please feel free to email me at any time.
© Partner400, 2015
Basic Syntax - XML-INTO variable
XML-INTO { ( E H ) } variable %XML ( ... )
• E and H extenders can be specified but there is little point
• variable is the target for the operation
✦
✦
It can be a field, an array, a Data Structure or a Data Structure array
If assignment is to a numeric field, the rules of %DEC conversion etc. apply
• %XML( ...) this BIF identifies the XML document
✦
It is also used to supply processing options
-
Which we will discuss later
Matching is based on names
• The actual names and hierarchy of the fields in the variable must match
the names and hierarchy of the elements and attributes in the XML
✦
More on this on the next chart
XML-INTO Customer
%XML(XML_Input1: 'case=any');
"Matching" Names and Hierarchy
This XML document
Can be processed using this DS
<Customer>
<Name> Smith </Name>
<City> Toronto </City>
</Customer>
...
...
...
<Customer>
<Name> Jones </Name>
<City> Chicago </City>
</Customer>
Dcl-Ds Customer
City
Name
Dim(99)
DIM( ) is used
for repeating elements
Note:
1. Names MUST match - More later on special
characters in names etc.
2. The field sequence in the DS does not have
to match the element sequence in the XML
3. Size of array is your choice
© Partner400, 2015
%XML BIF Details
The %XML BIF is used with both XML-INTO and XML-SAX
Syntax: %XML ( xml_document : options )
xml_document - Identifies the source of the XML
By default the XML is contained within the named variable
If option 'doc=file' is specified then the variable contains the IFS file name of the
XML document
options - Controls the processing of the XML stream
Format is optionname1=value1 optionname2=value2
No space allowed between the option name and the equal sign
Or between the equal sign and the value
Options can be specified in any case.
e.g. doc=file or DOC=FILE or Doc=File
Can be a literal string, a named constant or a character variable
We will look at some of the options as we use them
XML Used in Examples 2, and 3
<Customers>
<Company>Rinky Dinky Toy Co.</Company>
<Company>Partner400</Company>
<Company>BlackHole Navigation</Company>
<Company>Bankers Trust</Company>
</Customers>
This is the XML data that will be used in our initial examples
Although very few of the XML documents you see will be this simple!
The root element is Customers
It contains multiple Company name elements
This data is stored in variable XML_Input1
And in the IFS file /partner400/XML_Input1.xml
© Partner400, 2015
XML-INTO - Example 1 - Fill an Array
Dcl-Ds customers;
company Char(32) Dim(99);
End-Ds;
Dcl-Ds progStatus psds;
xmlElements Int(20) POS(372);
End-Ds;
Dcl-S i
Int(10);
XML-INTO company
%XML(XML_Input1: 'case=any' );
dsply ( %Char(xmlElements) + ' elements loaded' );
for i = 1 to xmlElements;
dsply company(i);
endfor;
Results
6 elements loaded
Phones R Us
Suchadeal Bank
Rinky Dinky Toy Co.
Partner400
BlackHole Navigation
Bankers Trust
Extracting into an array
• xmlElements is a new field that has been added at position 372 in the PSDS
RPG supplies a count of the elements that were loaded into the array
%XML Option case=any
• Indicates that element names in the XML can be in any case and should be
converted to upper case before comparing with the RPG variable names
Notes
Whenever the target of XML-INTO is an array, RPG ensures that a count of the number of elements processed in the
PSDS at offset 372. This is an 8 byte integer (20i) that can contain a really, really, big value! Quite why IBM made it
so big we can't say - but you're not ever going to exceed it.
Although we can use mixed case names for RPG variables (e.g. CustName, ZipCode, etc.) it is important to
understand that the RPG compiler reduces them all to upper case. So custName, CustName, and CUSTNAME are
all the same variable - CUSTNAME. As we noted earlier, the RPG variable names are matched against the XML
element names while processing the document, so unless the XML element names are all in upper case, we need to
specify the %XML option 'case=any' to force the parser to convert the element names to upper case before making
the comparison.
D-spec version of definitions:
d customers
d company
D progStatus
D
xmlElements
d i
© Partner400, 2015
DS
SDS
s
32a
Dim(99)
20i 0 Overlay(progStatus: 372)
10i 0
XML-INTO - Example 2 - XML in IFS
Dcl-Ds customers;
End-Ds;
Dcl-Ds progStatus psds;
xmlElements Int(20) Pos(372);
End-Ds;
Dcl-S XML_Input Varchar(256) Inz('/Partner400/XML_Input1.xml');
XML-INTO company
%XML(XML_Input: 'case=any doc=file ' );
This is the same basic example as the previous one
• But it demonstrates how to handle an XML document in the IFS
• Note the option 'doc=file'
%XML Option doc=file
•
Tells the compiler that the variable XML_Input contains the name of the file
containing the XML document
Notes
It surprises many people that the default behavior for XML-INTO is to assume that the XML document is contained
within the variable. After all, most people’s first introduction to XML is when someone tells them they have a file of
XML data that needs processing.
IBM’s assumption is that while this may be the case today, in a very short time, the majority of XML that you process
will arrive in your program as a result of a call to a web service. It will therefore be in a variable - and why force the
programmer to write it to a stream file just to read it back in again. Whether that assumption is proven correct or not
only time will tell.
D-spec version of data definitions:
D customers
D company
D progStatus
D
xmlElements
D XML_Input
D
© Partner400, 2015
DS
32a
Dim(99)
SDS
20i 0 Overlay(progStatus: 372)
s
256a
Varying
Inz('/Partner400/XML_Input1.xml')
Agenda
Beyond the Basics of XML-INTO
A more complex and realistic example
Additional capabilities in V6 and 7
Notes
As noted earlier - the example we have been looking at is far more simplistic than any we will encounter in real life.
In the following section we will look at how to process a more "normal" XML document and in particular the
importance of understanding how the Data Structures we will use to contain the data must be built.
© Partner400, 2015
XML For Example 3
<Customers>
<RecordCount> 2 </RecordCount>
<Customer type="wholesale">
<Address>
<Street>5678 High Street</Street>
<City>Mytown</City>
<State>GA</State>
<Zip>30033</Zip>
</Address>
</Customer>
<Customer type="retail">
<Contact>Meanold Miser</Contact>
<Address>
<Street>91011 Important Ave.</Street>
<City>Bankville</City>
<State>MN</State>
<Zip>55901</Zip>
</Address>
</Customer>
</Customers>
Notes
© Partner400, 2015
Example 3 - Compound Data Structure
Dcl-Ds customers Qualified;
recordCount Packed(3);
customer
LikeDS(customer_T) Dim(99);
End-Ds;
Dcl-Ds customer_T Qualified Template;
type
Char(10);
company
Char(32);
discountCode Char(1);
address
LikeDS(address_T);
End-Ds;
Dcl-Ds address_T Template;
street Char(32);
city
Char(24);
state
Char(2);
zip
Zoned(5);
End-Ds;
These nested Data Structures map the “shape” of the XML
Note that attributes of an element such as "type" are at the same level in
the hierarchy as nested elements such as "Company"
Notes
D-spec version for those not yet fully liberated:
Note the use of the keyword Template (V6+) to conserve storage. Since we will never store data in customer_T (the
data will actually be stored in customers.customer) why waste storage on it. Templates effectively allow us to define
our own data types - in this case a "customer" entry.
D customers
D recordCount
D customer
DS
D customer_T
D type
D company
D discountCode
D address
DS
D address_T
D street
D city
D state
D zip
DS
© Partner400, 2015
3p 0
Qualified
LikeDS(customer_T) Dim(99)
Qualified Template
10a
32a
1a
LikeDS(address_T)
Template
32a
24a
2a
5s 0
Example 3 - The Relationships
<Customers>
<RecordCount> ...
<Customer
type="wholesale">
<Contact> ...
<Company> ...
<Address>
<Street> ...
<City> ...
<State> ...
<Zip> ...
</Address>
</Customer>
<Customer
type="wholesale">
customers
Top level DS
recordCount
customer (1) Nested DS
type
company
discountCode
address
Nested DS
street
city
state
zip
customer (2) Nested DS
type
company
discountCode
address
Nested DS
street
city
state
zip
This is how the elements in the XML map to the various parts of the customers DS
Note that discountCode is present in the DS but not the XML
Similarly the element Contact is present in the XML but not the DS
Notes
This chart is designed to show how the nested structures map out in memory and how the DS relates to the XML
document we are processing.
© Partner400, 2015
Example 3 - Program Logic
XML-INTO customers
%XML(XML_Input : 'case=any allowextra=yes +
allowmissing=yes');
Two additional processing options are required for this document:
allowextra - Used when the document contains an element not in the DS
• The XML element "Contact" in this example
allowmissing - Used when the DS includes fields which may not be
present in the XML
• In other words fields that are optional in the XML document
• The field "discountCode" in this example is in the DS but not the XML
This particular XML requires "allowmissing" for another reason ...
• If there were less than 99 Customer elements, then any unused ones
would be considered to be "missing"
Notes
The biggest problem with these options is the lack of granularity. Ideally what we might want to say is "Allow
discountCode to be missing" or "Ignore the additional field Contact, we don't need that information".
But we can't do that - and that means that we really should only use these options when we have to.
AllowExtra is the least dangerous - and that's good because it is the one for which we have no cure. Using it simply
means that if additional elements are added to the document your RPG process will ignore them. If this is a situation
that should never occur and you want to know if it happens then _don't_ use this option.
AllowMissing presents a much more difficult situation because once you specify the option then even if every single
element that you wanted if missing from the document RPG will say it processed just fine. Until IBM released a
"cure" for this situation in V6 this was problematic as without the option, many genuine documents would fail.
Not just optional elements would cause this, but also any repeating elements that did not have the full number you
specified in the DIM present. For example if you allowed up to 5 addresses per customer in the DS ( i.e. Dim(5) )
and only 4 were present, then without the AllowMissing option the parsing would fail.
The "cure" to the AllowMissing issue was supplied via V6 and V7 PTFs and we will be looking at how that works on
the next chart.
© Partner400, 2015
Handling Optional Elements and More V6+
Optional Elements (A)
• In the case below Address can repeat but how many are there in the document?
✦
There may only be one or perhaps as many as ten
• Prior to this new option AllowMissing was the only solution
✦
✦
countprefix provides a versatile and much safer approach
You can now have RPG count exactly how many of a specified element are present
Data and attributes in the same element (B)
• Prior to option datasubf you had to parse the document twice
✦
Once to obtain the element data and once to obtain its attributes
• Now you can get all of the data in a single parse
<CustomerData>
<Customer>
(B) <Name category="Retail" ID="J020">Jones and Co.</Name>
(A) <Address type="Mailing">
<Street>2550 Main Street</Street>
<City>Knoxville</City>
There may be as few as 1
<State>TN</State>
Address - and as many as 10
</Address>
(A) <Address type="Shipping">
We need to keep track of
<Street>45 Opryland Drive</Street>
how many there are
.......
Notes
These new capabilities were added via a PTF issued in March of 2009
For full details see www.ibm.com/software/rational/cafe/docs/DOC-2975
We also wrote an article in the April 2009 edition of IBM Systems Magazine Extra Newsletter
www.ibmsystemsmag.com/ibmi/enewsletterexclusive/25012p1.aspx
Note that this is NOT available for V5R4 - only V6 and later
© Partner400, 2015
CountPrefix and DataSubf in Action
V6+
Dcl-Ds Customer Qualified Dim(99);
Name
LikeDS(Name_T);
Address
LikeDS(Address_T) Dim(10);
countAddress Int(5) Inz; // Contains the number of Address elements
End-Ds;
Dcl-Ds Name_T Template;
category Char(10);
ID
Char(4);
data
Char(32); // Contains data associated with the Name element
End-Ds;
Dcl-Ds Address_T Template;
type
Char(10);
Street Char(32);
City
Char(32);
State
Char(2);
End-Ds;
XML-INTO Customer
%XML(XMLFile: 'datasubf=data +
countprefix=count case=any');
In V5R4 ...
Name data could not have been retrieved without an additional processing pass
Determining number of Address elements required initialization with (say) hi-vals and then testing
for them after XML-INTO completed.
It also required specifying allowmissing=yes and would not work with %HANDLER
Notes
If you are not yet familiar with free-form syntax, here is the same code with fixed form D-specs:
D Customer
D
Name
D
Address
DS
Qualified Dim(99)
LikeDS(Name)
LikeDS(Address) Dim(10)
// countAddress contains the count of Address elements
D
countAddress
D Name
D
category
D
ID
5i 0 Inz
DS
10a
4a
Template
// data contains the data associated with the Name element
D
data
D Address
D
type
D
Street
D
City
D
State
32a
DS
Template
10a
32a
32a
2a
XML-INTO Customer
%XML(XMLFile: 'doc=file datasubf=data +
countprefix=count case=any');
© Partner400, 2015
Example 3 - Using CountPrefix
Dcl-Ds customers Qualified;
recordCount
Packed(3);
customer
LikeDS(customer_T) Dim(99);
count_customer Int(3);
End-Ds;
Dcl-Ds customer_T Qualified Template;
type
Char(10);
count_type
Int(3);
company
Char(32);
discountCode
Char(1);
count_discountCode Int(3);
address
LikeDS(address_T);
End-Ds;
Dcl-Ds address_T Template;
street Char(32);
city
Char(24);
state
Char(2);
zip
Zoned(5);
End-Ds;
XML-INTO customers
%XML(XML_Input: 'case=any allowextra=yes +
countprefix=count_' );
I "fixed" the earlier version of this example so that it now allows for:
• Less than 99 customers
• The type attribute can be omitted (attributes are often defaulted)
• The discountCode can be omitted
allowmissing for this document should no longer be needed
Notes
D-spec version for those not yet fully liberated:
This is the fixed form version of the declarations on the previous page.
D customers
DS
D recordCount
D customer
D count_customer
Qualified
3p 0
D customer_T
DS
D type
D count_type
D company
D discountCode
D count_discountCode
D address
D address_T
D street
D city
D state
D zip
© Partner400, 2015
DS
LikeDS(customer_T) Dim(99)
3i
Qualified Template
10a
3i
32a
1a
3i
32a
24a
2a
5s 0
LikeDS(address_T)
Template
Handling Namespaces
V6+
This new feature enables XML-INTO to handle namespaces
• There are also other fixes that really round out the XML-INTO support
What is a "Namespace" ?
• It ensures that elements in your XML document are uniquely named
✦
Vital to enable one document to be safely "wrapped" within another
• The namespace is associated with a URL thereby making it unique
✦
In the example the namespace P400 is associated with Partner400.com
• Element names are qualified with the namespace
✦
The same as RPG's qualified names but using a colon (:) not a period (.)
Very common in XML documents
• But until now RPG could not cope with it and you had to pre-process the
document
<P400:customer xmlns:P400="http://www.partner400.com">
<P400:address>
<P400:street>61 Kenninghall Cres.</P400:street>
<P400:city>Mississauga</P400:city>
<P400:postcode>L5N 2T8</P400:postcode>
</P400:address>
</P400:customer>
XML-INTO: Namespaces (V6 & V7)
V6+
%XML option ns={ nsOption }
• Controls the way in which XML-INTO handles a namespace
• Modifies element name before matching it against the names in the DS
Two possible values for the nsOption:
• remove
✦
Namespace prefix is ignored completely
• merge
✦
Replaces colon by an underscore to form a valid RPG name
• There is also the new nsprefix={prefix} option
✦
Allows retrieval of the namespace value when ns=remove is used
Note:
• if path is specified it must use names that result from the use of the ns= option
XML-Into P400_Address %XML(MyXMLDoc1 :
'path=P400_customer/P400_address doc=file ns=merge');
XML-Into Address %XML(MyXMLDoc1:
'path=customer/address doc=file +
ns=remove nsprefix=ns_');
© Partner400, 2015
XML-INTO: ns=remove Example
V6+
The namespace is being stripped
• So characters up to and including the colon are removed
nsprefix is used to access the original prefix value
• In this case we only wanted the prefix value for the Street element
<P400:customer xmlns:P400="http://www.partner400.com">
<P400:address>
</P400:address>
</P400:customer>
Dcl-Ds Address;
Street
Varchar(30);
ns_Street Varchar(20);
City
Varchar(25);
Postcode
Char(7);
End-Ds;
XML-Into Address %XML(MyXMLDoc1:
'path=customer/address doc=file +
ns=remove nsprefix=ns_');
Notes
D-spec version:
D Address
D Street
D ns_Street
D City
D Postcode
© Partner400, 2015
DS
30a
20a
25a
7a
Varying
Varying
Varying
XML-INTO: ns=merge Example
V6+
The namespace is merged with the element name
• The colon being replaced by an underscore
✦
Note the extended names are also used in the path= option
<P400:customer xmlns:P400="http://www.P400.com">
<P400:address>
</P400:address>
</P400:customer>
Dcl-Ds P400_Address;
P400_Street
Varchar(30);
P400_City
Varchar(25);
P400_Postcode Char(7);
End-Ds;
XML-Into P400_Address
%XML(MyXMLDoc1 :'path=P400_customer/P400_address
doc=file ns=merge');
Notes
D-spec version:
D P400_Address...
D
DS
D P400_Street...
D
D P400_City...
D
D P400_Postcode...
D
D ...
© Partner400, 2015
30A
Varying
25A
Varying
7A
XML-INTO - The Latest Updates
V6+
A fix for non-RPG characters in element names
• case=convert
This converts the following:
• Hyphens in XML names are converted to underscores
• Accented characters such as à, é and ñ convert to their uppercase A-Z equivalent
• Any remaining illegal characters are converted to underscores
If there is now an underscore at the beginning of a name it is dropped
• And multiple consecutive underscores are merged into a single underscore
Notes
One other issue that several RPG users have encountered concerns the use of characters in element names that
aren’t valid RPG field names. For example, it’s not uncommon for element names to include the hyphen but while
this character is legal in COBOL field names, it isn’t valid in RPG. As a result, such documents can’t be correctly
handled by XML-INTO and require alternative or additional processing. There’s an even bigger problem in European
countries, Mexico and the province of Quebec—to name a few. In these areas, many element names will include
accented characters such as à, é and ñ.
This is now handled by an extension to the group of case= options. Now in addition to the existing values ("upper,"
"lower" and "any") there’s now a "convert" option. When this is specified, any accented characters in a namespace,
element or attribute name are converted to their uppercase A-Z equivalent. If after that conversion any characters
remain that would be invalid in an RPG name, they’re converted to underscores. Should this conversion result in an
underscore at the beginning of a name, it’s simply dropped. If there are multiple consecutive underscores, these are
merged into a single underscore in the final name.
If you want to go into these new options in more detail we recommend reading the article we wrote on it for IBM
Systems Magazine
The title is "New and Improved XML-INTO"
www.ibmsystemsmag.com/ibmi/xml-into_namespace/35968p1.aspx
© Partner400, 2015
Handling Large Documents
Using XML-INTO with %Handler
XML-INTO Using %HANDLER
XML-INTO { ( E H ) } %HANDLER ( ... ) %XML ( ... )
Commonly used prior to V6 when size of data exceeded RPG's limits
• i.e. Number of elements exceeded 32,767 or the size of the DS exceeded 64K
– There are still limits at 6.1 and later - but they are much, much higher
Also useful if you want to process the XML in pieces
• For example one order at a time
Notice that %Handler replaces the INTO variable
Instead of storing the data directly XML-INTO passes it to you to process
%HANDLER BIF identifies the subprocedure that will "handle" the processing
• It will be called as each "INTO" action is completed.
• It can then process the data in any way that you like
• This is a technique known as "Call Back" processing
• It can also be very useful in your own applications
© Partner400, 2015
How the %Handler Process Operates
This is a "call-back" process
C functions such as qsort and bsearch use the same approach
XML-INTO (or -SAX) starts the process by calling the XML parser
The parser notifies the handler when it has data ready to process
The handler processes the data and returns control to the parser
The parser returns to step 2 unless parsing is completed at which time ...
The parser returns control to the main-line
RPG Main Line
Code
XML-INTO (or XML-SAX)
%HANDLER identifes handling
routine
Passes communications area
1
4
2
RPG Handler
Subprocedure
Parser
processes
XML
document
Receives data passed to it by the
parser
Plus the original communications
area
3
Notes
Call-back processing is a somewhat "alien" technique to RPG programmers although widely used in many other
languages. It is very useful in circumstances where you have a utility function (such as an XML parser!) but need to
customize its operation. Think of it as a bit like user exits in package software.
In addition to its use with the %HANDLER BIFs, call-back processing is used in many other utilities. For example the
sort and search functions qsort() and bsearch() both make use of call-back processing. You might find the
description of these functions in the RPG Redbook "Who Knew You Could Do That With RPG" useful if you find the
concept hard to grasp. You can find the Redbook here: www.redbooks.ibm.com/abstracts/sg245402.html
© Partner400, 2015
%HANDLER BIF - XML-INTO Version %HANDLER ( prototype : comm_area)
• The first parameter names the prototype of the handling procedure
✦
This procedure will be called as each "INTO" action completes
Handler's Parameter definition is shown below and described here:
• First parameter is the comm(unications) area
✦
Use it to communicate between your main program and the handler
-
i.e. To indirectly pass parameters to the handler subprocedure when it is not be in the
same program as the main line code
• Second parameter is the area to be filled ( i.e. It is the -INTO target )
✦
It must be an array - even if it only contains 1 element
• Third is an RPG supplied count of the number of array elements filled
✦
It will never be zero - at that point control will be back in your mainline code
Dcl-Pr MyHandler
commArea
company
numElements
End-Pr;
Int(10);
LikeDS(myCommArea);
Like(companyName) Dim(4) Const;
Int(10) Value;
Notes
Unless you already have a good grasp of call-back concepts, the use of %Handler can be hard to get your head
around. We have found that the best way to get a handle on the situation (sorry - couldn't resist!) is to run the
program in debug.
Step through your program making sure that you either set a breakpoint in your handler routine or use Step-into (not
just step) when you step the XML-INTO operation. You will find that you immediately pop-up in the handler
subprocedure. If you step through that code to the return, you will find immediately after the return operation you will
pop back to the beginning of the subprocedure. This seems a little odd since we would normally expect to return to
the point at which we were called. And of course that is just what happens - but we don't see it because the caller
was the XML parser! So after the return we went back to the parser - it did more work and then called us again - but
we didn't see that happen because we can't "see" inside IBM's code. Eventually the parser will have completed its
work, and that is when we will return to the point at which the XML-INTO (or indeed XML-SAX) was executed.
d MyHandler
d
commArea
d
company
d
numElements
© Partner400, 2015
PR
10i 0
LikeDS(myCommArea)
Like(companyName) Dim(4) Const
10i 0 Value
XML-INTO - Example 4 - %HANDLER
This is similar to Example 2 but uses %Handler
• This allows us to process documents in pieces
✦
Also helps deal with large XML documents
-
✦
Less important since the V6 relaxation of data size limitations
Each time the parser has X elements ready the handler routine is called
-
X being the number of elements we specify on the Dim in the prototype
It may be called one last time to handle the final group of elements
Dcl-Pr HandleCompany
commArea
company
numElements
End-Pr;
Int(10);
Int(10);
Char(32) Dim(4) Const;
Int(10) Value;
Dcl-Ds customers;
End-Ds;
Dcl-S count Int(10) Inz;
XML-INTO %Handler( HandleCompany: count )
%XML(XML_Input1: 'path=Customers/Company case=any' );
Dsply ('Processed ' + %char(count) + ' total elements');
Notes
d HandleCompany
d
commArea
d
company
d
numElements
PR
d customers
d company
DS
d count
S
© Partner400, 2015
10i 0
10i 0
32a
Dim(4) Const
10i 0 Value
32a
Dim(4)
10i 0 Inz
XML-INTO - Example 4 - The handler
numElements contains the count of elements passed on this call
• It can only ever be less than the maximum on the final call
✦
And it will never be zero
Dcl-Proc HandleCompany;
Dcl-Pi *N
count
company
numElements
End-Pi;
Dcl-S i
Int(10);
Int(10);
Char(32) Dim(4) Const;
Int(10) Value;
Int(10);
for i = 1 to numElements;
dsply company(i);
endfor;
// add current count to total in Comm Area and display current count
count += numElements;
dsply ( %Char(numElements) + ' elements loaded on this call' );
return 0;
End-Proc;
Results of %Handler Program
<Customers>
<Company>Rinky Dinky Toy Co.</Company>
<Company>Partner400</Company>
<Company>BlackHole Navigation</Company>
<Company>Bankers Trust</Company>
</Customers>
Sample XML contained the above customer entries, and the array had a maximum capacity of 4.
This is the result of running our example program against that XML stream.
DSPLY Phones R Us
DSPLY Suchadeal Bank
DSPLY Rinky Dinky Toy Co.
DSPLY Partner400
DSPLY 4 elements loaded on this call
DSPLY BlackHole Navigation
DSPLY Bankers Trust
DSPLY 2 elements loaded on this call
DSPLY Processed 6 total elements
D-spec version of the definitions:
p
d
d
d
d
HandleCompany
count
company
numElements
d i
© Partner400, 2015
b
pi
s
10i 0
10i 0
32a
Dim(4) Const
10i 0 Value
10i 0
XML-SAX
XML-SAX
The basics
XML-SAX
Dcl-Pr myHandler Int(10);
commArea LikeDS(myCommArea);
event Int(10) VALUE;
pstring Pointer VALUE;
stringLen Int(20) VALUE;
exceptionId Int(10) VALUE;
End-Pr;
"SAX" stands for Simple API for XML
But there's no such thing as a truly "Simple" example of SAX parsing
So we will simply demonstrate the basic process of identifying events
With XML-SAX the %HANDLER BIF must always be specified
Parameters to the Handler routine are not the same as for XML-INTO
Except for the return value and the Communications Area
Second parameter is a numeric value to notify you of which event occurred
‣ It can be compared with RPG IV Special Words such as *XML_ATTR_NAME
The third is a pointer to the string containing the current token
‣ i.e. The attribute or element name, or data content
– The fourth is the length of that string
– Fifth contains an exception Id. when event *XML_EXCEPTION is signaled
© Partner400, 2015
XML-SAX Events
These include:
*XML_START_DOCUMENT
Indicates that parsing has begun
*XML_START_ELEMENT
The name of the XML element that is starting
*XML_CHARS
The value of the XML element
*XML_END_ELEMENT
The name of the XML element that is ending
*XML_ATTR_NAME
The name of the attribute
*XML_ATTR_CHARS
The value of the attribute
*XML_END_ATTR
Indicates the end of the attribute
*XML_END_DOCUMENT
– Indicates the end of the document
Notes
This is just a small sampling of XML-SAX event codes. There are more than 20 in total although many you will never
see in practice. The RPG manual details all of the codes and their meanings.
© Partner400, 2015
XML-SAX Example
Dcl-Pr MySAXHandler
commArea
event
pstring
stringLen
exceptionId
End-Pr;
Int(10);
Like(myCommArea);
Int(10) Value;
Pointer Value;
Int(20) Value;
Int(10) Value;
Dcl-Ds eventCodeData;
*N
Int(10) Inz(*XML_START_DOCUMENT);
*N
Int(10) Inz(*XML_VERSION_INFO);
// ...
eventCodes Int(10) Dim(25) POS(1);
End-Ds;
XML-SAX %Handler( MySAXHandler : myCommArea )
%XML( 'XMLSAXDOC.xml' : 'doc=file');
As noted a simple SAX example is a contradiction
So our example will simply demonstrate the basic mechanism
It is part of a program that reports on the content of an XML document
Full source code is here: www.Partner400.com/Examples/XMLSAXTST.HTM
Notes
D-specs version:
D MySAXHandler
D
commArea
D
event
D
pstring
D
stringLen
D
exceptionId
Pr
d eventCodeData
d
d
d
d eventCodes
DS
© Partner400, 2015
10I 0
10I 0
*
20I 0
10I 0
10i
10i
10i
10i
0
0
0
0
Like(myCommArea)
Value
Value
Value
Value
Inz(*XML_START_DOCUMENT)
Inz(*XML_VERSION_INFO)
Inz...
Dim(25) Overlay(eventCodeData)
XML-SAX Processing
Dcl-S
Dcl-S
Dcl-S
Dcl-S
string
UCS2String
element
eventIndex
Char(65535) Based(pstring);
Varucs2(16383) Based(pString);
Int(10);
Int(5);
eventIndex = %lookup( event : eventCodes );
// identify event
if eventIndex > 0;
eventName = eventNames( eventIndex );
if event = *XML_EXCEPTION;
data = '*** Exception offset is ' + %char(stringLen);
returnCode = -1;
elseif stringLen > 0;
// Use string length - ptr is unreliable
if ( event = *XML_UCS2_REF OR event = *XML_ATTR_UCS2_REF );
UCS2StringLen = ( stringLen / 2 );
data = %char( %subst(UCS2String : 1 : UCS2StringLen ) );
else;
data = RmvWhiteSpace(%subst(string : 1 : stringLen));
endif;
else;
data = '** No Data **';
endif;
else;
// Unknown event
...
Notes
D-spec version of definitions:
D string
s
65535a
D UCS2String
D element
s
s
16383a
Based(pString)
10i 0
D eventIndex
s
© Partner400, 2015
Based(pstring)
5i 0
The Critical Part !
...
elseif stringLen > 0;
// Use string length - ptr is unreliable
if ( event = *XML_UCS2_REF OR event = *XML_ATTR_UCS2_REF );
UCS2StringLen = ( stringLen / 2 );
data = %char( %subst(UCS2String : 1 : UCS2StringLen ) );
else;
data = RmvWhiteSpace(%subst(string : 1 : stringLen));
...
Use the length of the string to determine if there is any data
Manual indicates that the pointer is null when there is no data - but it lies!
And also use the length to control the extraction of the data
Remember that the field string is big (65,535 characters) and that only the first part
is valid - that’s why I used %SUBST
Notes
D-specs:
D
D
D
D
string
UCS2String
element
eventIndex
© Partner400, 2015
S
S
s
s
65535A
16383C
10i 0
5i 0
Based(pstring)
Based(pString)
Sample Report
This is an extract from the report produced by the program when
processing a sample XML document.
• Items in RED cannot be retrieved by XML-INTO only by XML-SAX
Event Code and RPG name
20 *XML_START_DOCUMENT
25 *XML_VERSION_INFO
10 *XML_ENCODING_DECL
18 *XML_STANDALONE_DEC
9 *XML_DOCTYPE_DECL
6 *XML_COMMENT
21 *XML_START_ELEMENT
5 *XML_CHARS
21 *XML_START_ELEMENT
2 *XML_ATTR_NAME
4 *XML_ATTR_CHARS
26 *XML_END_ATTR
2 *XML_ATTR_NAME
23 *XML_UNKNOWN_ATTR_R
26 *XML_END_ATTR
13 *XML_END_ELEMENT
Length
13
8
3
43
34
8
1
5
4
12
18
3
15
Data
"** No Data **"
"1.0"
"ibm-1140"
"yes"
"<!DOCTYPE page [ <!ENTITY abc "ABC"> ]>"
"This document is just an example"
"sandwich"
""
"bread"
"type"
"baker's best"
"** No Data **"
"supplier"
"abc"
"** No Data **"
"bread"
Questions ????
E-mail me at
Any time!
© Partner400, 2015
RPG XML-xxxx Error Status Codes
1: The parser found an invalid character while scanning white space outside element content.
2: The parser found an invalid start of a processing instruction, element, comment, or document type declaration
outside element content.
3: The parser found a duplicate attribute name.
4: The parser found the markup character '<' in an attribute value.
5: The start and end tag names of an element did not match.
6: The parser found an invalid character in element content.
7: The parser found an invalid start of an element, comment, processing instruction, or CDATA section in element
content.
8: The parser found in element content the CDATA closing character sequence ']]>' without the matching opening
character sequence '<![CDATA['.
9: The parser found an invalid character in a comment.
10:The parser found in a comment the character sequence '--' (two hyphens) not followed by '>'.
11: The parser found an invalid character in a processing instruction data segment.
12: A processing instruction target name was 'xml' in lowercase, uppercase or mixed case.
13: The parser found an invalid digit in a hexadecimal character reference (of the form &#xdddd;, for example
&#x0eb1).
14: The parser found an invalid digit in a decimal character reference (of the form &#dddd;).
15: A character reference did not refer to a legal XML character.
16: The parser found an invalid character in an entity reference name.
17: The parser found an invalid character in an attribute value.
18: The parser found a possible invalid start of a document type declaration.
19: The parser found a second document type declaration.
20: An element name was not specified correctly. The first character was not a letter, '_', or ':', or the parser found an
invalid character either in or following the element name.
21: An attribute was not specified correctly. The first character of the attribute name was not a letter, '_', or ':', or a
character other than '=' was found following the attribute name, or one of the delimiters of the value was not correct,
or an invalid character was found in or following the name.
22: An empty element tag was not terminated by a '>' following the '/'.
23: The element end tag was not specified correctly. The first character was not a letter, '_', or ':', or the tag was not
terminated by '>'.
24: The parser found an invalid start of a comment or CDATA section in element content.
25: A processing instruction target name was not specified correctly. The first character of the processing instruction
target name was not a letter, '_', or ':', or the parser found an invalid character in or following the processing
instruction target name.
26: A processing instruction was not terminated by the closing character sequence '?>'.
27: The parser found an invalid character following '&' in a character reference or entity reference.
28: The version information was not present in the XML declaration.
29: The 'version' in the XML declaration was not specified correctly. 'version' was not followed by '=', or the value was
missing or improperly delimited, or the value specified a bad character, or the start and end delimiters did not match,
or the parser found an invalid character following the version information value closing delimiter in the XML
declaration.
30: The parser found an invalid attribute instead of the optional encoding declaration in the XML declaration.
31: The encoding declaration value in the XML declaration was missing or incorrect. The value did not begin with
lowercase or uppercase A through Z, or 'encoding' was not followed by '=', or the value was missing or improperly
delimited or it specified a bad character, or the start and end delimiters did not match, or the parser found an invalid
character following the closing delimiter.
© Partner400, 2015
32: The parser found an invalid attribute instead of the optional standalone declaration in the XML declaration.
33: The 'standalone' attribute in the XML declaration was not specified correctly. 'standalone' was not followed by a
'=', or the value was either missing or improperly delimited, or the value was neither 'yes' nor 'no', or the value
specified a bad character, or the start and end delimiters did not match, or the parser found an invalid character
following the closing delimiter.
34: The XML declaration was not terminated by the proper character sequence '?>', or contained an invalid attribute.
35: The parser found the start of a document type declaration after the end of the root element.
36: The parser found the start of an element after the end of the root element.
300: The parser reached the end of the document before the document was complete.
301: The %HANDLER procedure for XML-INTO or XML-SAX returned a non-zero value, causing the XML parsing to
end.
302: The parser does not support the requested CCSID value or the first character of the XML document was not '<'.
303: The document was too large for the parser to handle. The parser attempted to parse the incomplete document,
but the data at the end of the document was necessary for the parsing to complete.
500-999: Internal error in the external parser. Please report the error to your service representative.
10001-19999: Internal error in the parser. Please report the error to your service representative.
Limitations of the XML Parser
If the parsing is done in a single-byte character CCSID, the maximum number of characters that the parser can
handle is 2,147,483,408.
If the parsing is done in UCS-2, the maximum number of UCS-2 characters that the parser can handle is
1,073,741,704.
The parser does not support every CCSID. If your job CCSID is one of the CCSIDs that the parser does not handle,
you must parse your document in UCS-2.
The following EBCDIC CCSIDs are supported: 1047, 37, 1140, 273, 1141, 277, 1142, 278, 1143, 280, 1144, 284,
1145, 285, 1146, 297, 1147, 500, 1148, 871, and 1149.
The following ASCII CCSIDs are supported: 819, 813, 920.
The following Unicode CCSIDs are supported: 1200, 13488, 17584.
The parser does not support entity references. When it encounters an entity reference, it generates either an
"unknown reference" or "unknown attribute reference" event. The value of the event is the reference in the form
"&name;".
The parser does not parse the DOCTYPE declaration. The text of the DOCTYPE declaration is passed as the data
value for the "DOCTYPE declaration" event.
The parser does not generate "start prefix mapping" and "end prefix mapping" events. It ignores the colons in XML
element and attribute names.
© Partner400, 2015
"Instant" Web Services and
Stored Procedures
Jon Paris
www.Partner400.com
Notes
Jon Paris is co-founder of Partner400, a firm specializing in customized education and mentoring services for IBM i
* developers.
Jon's career in IT spans 40+ years including a 12 year period with IBM's Toronto Laboratory.
Together with his partner Susan Gantner, Jon devotes his time to educating developers on techniques and
technologies to extend and modernize their applications and development environments. Together Jon and Susan
author regular technical articles for the IBM publication, IBM Systems Magazine, IBM i edition, and the companion
electronic newsletter, IBM i EXTRA. You may view articles in current and past issues and/or subscribe to the free
newsletter at: www.IBMSystemsMag.com.
Jon and Susan also write a weekly blog on Things "i" - and indeed anything else that takes their fancy. You can
find the blog here: ibmsystemsmag.blogs.com/idevelop/
Feel free to contact the author at:
* IBM i - aka AS/400, System i, iSeries, i5, etc.
© Partner400, 2015
Agenda
How?
•
XMLSERVICE - the latest IBM toolkit
What is it ?
•
And why did we need a new toolkit ?
What is different about it ?
•
It is Open Source and written in RPG!
How is it used ?
•
From other languages and platforms
XMLSERVICE - What Is It ?
Created as part of the Zend/IBM partnership
•
Now the core of Zend's PHP Toolkit for IBM i
Allows usage of any program or utility on your IBM i
•
Programs, Service Programs, CL commands, PASE utilities, ...
From anywhere
•
•
Another IBM i
Windows, Unix, Linux, ...
Once set up nothing else required
•
New programs can be accessed without any additional work
Two forms of access supported
•
•
Web Service
Stored Procedure
Development site hosted by the Young i Professionals
•
youngiprofessionals.com
© Partner400, 2015
Why Did We Need a New Toolkit ?
Limitations in existing tooling
•
Java toolbox does not support return values
✦
✦
•
Except for four byte integers
** Limitation can be avoided in V7 by using the new RTNPARM option
Requires use of Java by requesting platform
✦
Alternatively the other platform needs a Java bridge
-
•
•
Java programming skills required to enhance toolbox
Slow to deploy new functionality
✦
•
If Java programmers don't need it there was no incentive to add it
New functionality means changes in PCML generation
✦
•
In all other cases you are limited to ODBC capability
So you have to wait for the compiler team to implement it
The result?
✦
It took years to get new data types added to the interface
Why Did We Need a New Toolkit ?
Limited support for persistence
•
Problematic if you want to deploy an existing 5250-style
application as a web service or browser application
Proprietary Technology
•
Java toolkit originally developed by IBM
✦
•
Subsequently released as Open Source
i5xxx components of PHP toolkit actually produced by Aura
✦
Known by Aura as EasyCom
Manual effort required to deploy new functionality
•
Have to set up a new web service or stored procedure or ...
© Partner400, 2015
What is Different About the New Toolkit ?
Open Source under BSD License
•
Completely free to use
100% RPG IV
•
Any RPG programmer can modify
All communications via XML documents
•
•
Easier to debug
Automatic conversion between ASCII and EBCDIC
Usable in single and two tier environments
•
•
All code on single IBM i
Or any other system to IBM i
Supports persistent connections
•
Called RPG programs can retain state information
✦
•
i.e. Just as they normally do in a 5250 job
Facilitates repurposing of existing 5250 applications as web
services or browser jobs
What is Different About the New Toolkit ?
Supports complex parameter types
•
Arrays, DS, DS Arrays, Compound DS, ...
Program calls and commands can be run in sequence
•
Ensures that results of one operation are available to the next
Directly supports return values
•
Although RPG's V7 RTNPARM keyword also allows you to avoid
this problem
Can be invoked from any Client-side language
•
•
As long as they can invoke a web service
Or use SQL connections
Security
•
Requires use of valid user name and password
✦
•
No matter what protocol is used
Facilitates access control to IBM i resources
© Partner400, 2015
Toolkit Architecture
Cross Platform
Language Agnostic
•
Accessible from any language
How Is It Used ? - From PHP
New Zend Toolkit uses XMLSERVICE under the covers
•
Supports all previous functionality
✦
•
Completely new set of APIs
✦
•
Except Record Level Access
Plus "wrappers" to allow existing code to use the new APIs
Delivered as part of the Zend Server download
✦
Or obtain from Young i Professionals site if you don't want PHP
•
http://youngiprofessionals.com/wiki/XMLSERVICE
Allow PHP access from other platforms
•
•
So you can develop complete PHP applications that invoke IBM i
functionality on Linux, Windows and Mac
The Zend components are also Open Source
All of the new APIs use an OO approach
•
Still very easy to use for the non-OO programmer
Details later if we have time
© Partner400, 2015
XMLSERVICE: Stored Procedure Interface
Supplied stored procedures are the preferred access method
•
Can be accessed via ODBC, JDBC, DB2Connect, etc.
Can be used to access multiple programs
•
Eliminates need to create one stored procedure per program
Accepts four parameters
•
IPC Key
✦
A path to an IFS folder associated with a specific XMLSERVICE job
-
•
CTL
✦
Parameters to control the service job's behavior
-
•
This is part of the persistence mechanism
e.g Debug mode, shut down job, etc.
CLOB In and CLOB out
✦
The Input and Output XML documents
XMLSERVICE: REST Web Service Interface
Takes Seven Parameters
✦
Note that three are the same as for the Stored Procedure interface
db2
•
Name of database to access (*Local or Database name)
uid
•
User Id
pwd
•
Password
ipc and ctl
•
Same as for the IPC and CTL Stored Procedure parameters
xmlin
•
XML Input document
xmlout
•
Maximum expected size of returned XML output document
© Partner400, 2015
XML Document Elements
<cmd> … </cmd>
•
Executes the command line command specified between the <cmd> tags
<sh> … </sh>
•
PASE equivalent of the <cmd> tag.
<pgm> … </pgm>
•
Identifies program or subprocedure to be run
✦
✦
Attributes name= lib= and func= identify program name, library, and subprocedure name
Parameters and return values for the program are defined within the <parm> element
<parm> … </parm>
•
Identifies a parameter. More details on next chart
<ds> … </ds>
•
Indicates that the parameter is a data structure. More details on next chart
<data> … </data>
•
Defines the parameter and populates it.
<return> … </return>
•
Identifies the return value on a subprocedure.
✦
Definition of the return value is via <data> and <ds> tags just like parameters
XML Document Element Attributes
<parm> attributes
•
•
io=... Parameter type "in" "out" or “both”
by=... Parameter passing method
✦
“val” (value) or “ref” (reference)
<ds> attributes
•
•
dim=... Maximum number of DS array elements
dou=... Number of array elements actually populated.
<data> attributes
•
•
var=... Names the variable
type=... Data type and size
✦
•
e.g. 12a, 15p5, 9s2, etc.
varying=... *on indicates a varying length field.
© Partner400, 2015
Other Features of the XML Documents
A single XML document can contain multiple requests
• They will be performed in sequence
• Makes it simple to perform tasks like setting the library list
Success or failure is indicated in the returned document
• See example in document below
Tags and attributes not used by XMLSERVICE are allowed
• They are simply passed through and included in the XML output
• You can add extra markup to assist in parsing returned XML
• Or comments that will aid in debugging
<?xml version="1.0"?>
<mult>
<cmd comment='addlible'>+++ success ADDLIBLE LIB(JONSLIB)</cmd>
<pgm name='MYPGM' lib='JONSLIB' func='MYPGM'>
<parm .....
Simple Example of the Web Service Interface
A simple program that takes two parameters
• A 15 digit packed numeric and a 30 character
• The second parameter will be modified by the called program
The User of your web service must send the XML shown below
// Proto for XMLSRVTST1
d XMLSrvTst1
pr
ExtPgm(‘XMLSRVTST1’)
d input
15p 5
d output
30a
< ?xml version='1.0'? >
< pgm name='XMLSRVTST1' lib='XMLSERVICE' >
< parm >
< data type='15p5' >5678901234.54321< /data >
< /parm >
< parm >
< data type='30A' >Original value< /data >
< /parm >
< /pgm >
© Partner400, 2015
This is the XML
representation of
the prototype
shown above
XML Returned by the Web Service
The document returned is almost identical to the one we sent
• But the parameter details (length and type) are omitted
• And the contents of the second parm have changed
✦
This is highlighted below
More on the XML in a moment
// Proto for XMLSRVTST1
d XMLSrvTst1
pr
ExtPgm(‘XMLSRVTST1’)
d input
15p 5
d output
30a
// Relevant portion of the XML document returned
< ?xml version='1.0'? >
< pgm name='XMLSRVTST1' lib='XMLSERVICE' >
< parm >
< data >5678901234.54321< /data >
< /parm >
< parm >
< data >Input was 5678901234.54321< /data >
< /parm >
< /pgm >
Original field definitions
are not returned in the
response document.
Only the field values
More Complex XML Call Document
Elements and attributes not required by XMLSERVICE are echoed
• Useful in debugging returned data
✦
Note the use of the field names
D LookupTest
D
searchFor
D
max
D
count
Pr
LikeDS(rec_T) Dim(10)
10a
10i 0
10i 0
<pgm name='MYSRVPGM' func='LOOKUPTEST'>
<parm comment='search for this name'>
<data var='searchFor' type='10A'>Paris</data>
</parm>
<parm comment='max allowed return'>
<data var='max' type='10i0'>5</data>
</parm>
<parm comment='count of records returned'>
<data var='count' type='10i0' enddo='count'>0</data>
</parm>
<return>
<ds var='rec_t' dim='10' dou='count'>
<data var='name' type='30A'>na</data>
<data var='job' type='40A'>na</data>
<data var='pay' type='9p2'>0.0</data>
</ds>
</return>
</pgm>
© Partner400, 2015
Document Returned From Procedure Call
<pgm name='MYSRVPGM' func='LOOKUPTEST'>
<parm comment='search for this name'>
<data var='searchFor'>Paris</data>
</parm>
<parm comment='max allowed return'>
<data var='max'>5</data>
</parm>
<parm comment='count of records returned'>
<data var='count'>3</data>
</parm>
<return>
<ds var='rec_t'>
<data var='name'>Jon Paris</data>
<data var='job'>Chief Cook and Bottle Washer</data>
<data var='pay'>13.42</data>
</ds>
<ds var='rec_t'>
<data var='name'>Susan Gantner-Paris</data>
<data var='job'>Manager</data>
<data var='pay'>26.84</data>
</ds>
...
</return>
</pgm>
Original field definitions
are not returned in the
response document.
Only the field values
More info and examples of PHP
usage follow but I do not have time to
go into them in detail.
Contact me in the break or by email
after the event if you have questions.
© Partner400, 2015
The New XMLService Web Demo Page
Nothing but HTML!
•
Honestly
Not pretty - but pretty amazing nonetheless
•
Examples of SQL, calling programs, issuing commands, etc.
youngiprofessionals.com/wiki/uploads/XMLSERVICE/
DemoHtml.html
•
Use browser's View Source to see the underlying HTML/XML
Example of HTML Used
This is the HTML code used to call the subprocedure ZZTIMEUSA
• Note how the XML data is defined

<form name="input" action="/cgi-bin/xmlcgi.pgm" method="post">
<input type="hidden" name="db2" value="*LOCAL">
<input type="hidden" name="uid" value="*NONE">
<input type="hidden" name="pwd" value="*NONE">
Note that the XML
<input type="hidden" name="ipc" value="/tmp/rangerhtmlonly">
packet is specified as the value
<input type="hidden" name="ctl" value="*sbmjob">
for the xmlin variable
<input type="hidden" name="xmlin" value=
"<?xml version='1.0'?>
<?xml-stylesheet type='text/xsl' href='/wiki/uploads/XMLService/DemoXslt.xsl'?>
<script>
<pgm name='ZZSRV' lib='XMLSERVICE' func='ZZTIMEUSA'>
<parm io='both'>
<data type='8A'>09:45 AM</data>
</parm>
<return>
<data type='8A'>nada</data>
</return>
</pgm>
It ends here
</script>">
<input type="hidden" name="xmlout" value="5000">
<input type="submit" value="call *SRVPGM (ZZTIMEUSA)" />
</form>
© Partner400, 2015
PHP Methods and Functions in ToolkitService
The main class that manages all DB2 settings and communication
•
•
Includes methods to Set Database Connections
And Set XML Service Options
CLCommand
•
Run CL commands either in interactive or batch mode
PGMCall method
•
Calls programs (*PGM) or service programs (*SRVPGM)
AddParameterxxxx methods
•
•
Define the program parameters.
Methods available for all the usual suspects
SetParameterValue
•
Sets new parameter values
disconnect interface
•
•
Terminates an XML service job in the ZENDSVR subsystem
Don't use this with persistent jobs unless and until you want to
PHP Usage - Issue Interactive CL Command
Copy User Id, and Password
plus default library etc.
<?php
include_once 'authorization.php';
include_once '../API/ToolkitService.php';
Copy in toolkit script
$obj = ToolkitService::getInstance($db, $user, $pass);
$parms = array('InternalKey'=>"/tmp/$user",'debug'=>true,'plug' => "iPLUG32K");
$obj->setToolkitServiceParams($parms);
$cmd = "addlible ZENDSVR";
$obj->CLCommand($cmd);
Initial CL Command is
executed followed by
the interactive
command below
Controls the maximum
size of message that
can be handled
$Rows = $obj->CLInteractiveCommand("DSPLIBL");
if($Rows)
Interactive command
returned a result set var_dump($Rows);
the 5250 display data
else
echo $obj->getLastError();
$obj->disconnect();
?>
© Partner400, 2015
PHP Usage - Call RPG Program
<?php
include_once 'authorization.php';
include_once '../API/ToolkitService.php';
$extension='ibm_db2';
$ToolkitServiceObj = ToolkitService::getInstance($db, $user, $pass, $extension);
$ToolkitServiceObj->setToolkitServiceParams(array('InternalKey'=>"/tmp/$user"));
$code = $_POST ['code']; // Get value of Code field from input
$desc = ' ';
// Blank the output parameter
Set up the parameters
$param[] = $ToolkitServiceObj->AddParameterChar('both', 10,'CODE', 'CODE', $code);
$param[] = $ToolkitServiceObj->AddParameterChar('both', 10,'DESC', 'DESC', $desc);
$result = $ToolkitServiceObj->PgmCall("TESTPGM", "ZENDSVR", $param, null, null);
if($result)
var_dump($result['io_param']);
else
echo "Execution failed.";
$ToolkitServiceObj->disconnect();
RPG program requires
two parameters each 10
characters long.
The previous lines placed
them in the array
?>
PHP Usage - Call Service Program Procedure
<?php
...
$SysValueName = "QCCSID";
$param[] = $ToolkitServiceObj->AddParameterChar('both', 1,'ErrorCode','errcode',
$Err);
$param[] = $ToolkitServiceObj->AddParameterChar('both', 10,'SysValName','sysvalname',
$SysValueName);
$param[] = $ToolkitServiceObj->AddParameterChar('both', 1024,'SysValue','sysvalue',
$SysValue);
$OutputParams = $ToolkitServiceObj->PgmCall('ZSXMLSRV', "ZENDSVR", $param, NULL,
array('func'=>'RTVSYSVAL') );
Function
name
if( isset($OutputParams['io_param']['sysvalname']))
echo " System value ".$SysValueName." = ".$OutputParams['io_param']['sysvalue'];
else
Change parameter
echo "System value $SysValueName was not retrieved.";
value
// Change requested value and execute call again
ProgramParameter::UpdateParameterValues( $param, array("sysvalname"=>"QLANGID"));
$OutputParams = $ToolkitServiceObj->PgmCall('ZSXMLSRV', "ZENDSVR", $param, NULL,
array('func'=>'RTVSYSVAL') );
...
?>
© Partner400, 2015
iToolkitClass Methods and Functions - Part 1
Data Queue
•
•
•
•
•
•
CreateDataQ()
DeleteDQ()
ReceieveDataQueue()
SendDataQueue()
SetDataQName()
ClearDQ()
User Space
•
•
•
•
•
•
CreateUserSpace()
RetrieveUserSpaceAttr()
RetrieveUserSpaceSize()
DeleteUserSpace()
WriteUserSpace()
ReadUserSpace()
iToolkitClass Methods and Functions - Part 2
Spooled Files
•
•
GetSPLList()
GetSPLF()
System Values
•
•
SystemValuesList()
GetSystemValue()
Job Log
•
•
•
JobList()
JobLog()
GetJobInfo
Object List
•
getObjectList()
© Partner400, 2015
Questions ?
If you think of any questions after
the session please email me:
© Partner400, 2015
© Partner400, 2015
RPG in V7
Including the New
Free-Form Definitions.
Jon Paris
www.Partner400.com
Notes
If you thought that RPGIV had changed over the years - well "You ain't seen nothin' yet!!"
V7 brought with it a number of enhancements but the one that has made the biggest impact has undoubtedly been the
introduction of free-form data and file definitions.
Most of us have been making extensive use of Evals since they were first introduced with RPG IV in V3R1. Perhaps like
us you have experienced frustration when forced to split a line because you had run out of space! Maybe you were even
tempted to shorten that nice meaningful variable name to avoid having to use a second line! While you were
contemplating this dilemma, you might have also noted the fact that there was a huge area of white space to the left of
the Eval you couldn't use.
V5R1 put an end to that frustration by introducing the notion of completely free-format logic specs. Coupled with a large
number of new Built-In Functions (BIFs) this "New" RPG remains familiar, while offering some very powerful new
capabilities.
Then in V7.1 (with TR7) almost all the rest of the RPG language went free-format - with the addition of free-format
replacements for H, F, D and P specs.
In this session we will look at:
How to code free format RPG logic
✦
How to replace operation codes that aren't supported in free format RPG
✦
The new BIFs that add power to the language
✦
New functions that only work in Free-form form logic
How to use the free format replacements for H, F, D and P specs
© Partner400, 2015
V 7.1 RPG Enhancements
Data Definition and Usage Enhancements
Sort and Search Data Structure Arrays
Support For Alias Names
New Scan and Replace BIF
Subprocedure related enhancements
Prototypes are optional
Performance improvements for large return values
Process Stored Procedure Result Sets
And of course free-form
file and data definitions!
Notes
We don't have time to go into all of these features in depth but we have written articles on many of them and of course
you can find more about any that I skip by checking out the V7 RPG manual in the Information Center.
© Partner400, 2015
Sorting Data Structure Arrays
Data structure arrays can now be sorted
• Using any of the subfields as the key
✦
But only a single subfield can be used
-
No direct support for sorting multi-dimensional arrays
• Asterisk (*) identifies the level at which the array is to be sorted
SORTA can now have sequence specified
• Using the Op-code extenders A(scending) or D(escending)
• Can only be used when no sequence specified in the D-specs
D products1
D
productCode
D
description
D
totalSales
D
qtyInStock
DS
Dim(999) Qualified
5a
40a
Varying
9p 2
5p 0
SortA products1(*).totalSales;
SortA products1(*).description;
SortA(A) products1(*).totalSales; // Sort ascending sequence
SortA(D) products1(*).description; // Sort descending sequence
Notes
With the advent of V5R2, it became possible to define Data Structure arrays. i.e. with a DIM keyword at the DS level.
But at the time IBM did not provide any means by which such arrays could effectively be sorted. To do that you had to
resort to using the qsort function. That shortcoming is removed in V7 and you can now sort on any subfield in the array.
For example, given the DS array here, you can perform a sort on qtyInStock or totalSales or any of the other fields in
the DS. As you can see from this code, the level of the array to be sorted is indicated by an asterisk (*) in the subscript
position.
In the first example the DS array is sequenced on the totalSales values, and in the second the description. Another nice
addition to the SORTA repertoire is that you can now specify whether the sort is to be in ascending or descending
sequence. Previously this was determined by the ASCEND or DESCEND keyword on the array definition and SORTA
used the defined sequence - which of course meant that without playing games (re-mapping the array via pointers etc.)
any given array could only ever be in ascending _or_ descending order. Now the op-code extenders (A) and (D) can be
used to specify the sequence in which the array is sorted.
The * is used to indicate the level at which the sorting should occur. Of course, in these examples, it's pretty obvious,
since it's the only level where sorting is possible. But this sorting capability also works with nested Data Structures, so
even very complicated structures can be sorted. The next chart shows you how.
© Partner400, 2015
Sorting Nested Data Structure Arrays
As noted earlier - no direct RPG support exists
• But this is one way to do it
D products2
D
productCode
D
description
D
salesByMonth
D
D
qtyInStock
DS
D salesByMonth
D
monthNumber
D
sales
DS
D i
S
5a
40a
5p 0
Dim(999) Qualified
Varying
LikeDS(salesByMonth)
Dim(12)
3p 0
9p 2
5i 0
// Sort each entry in turn into sales value sequence
For i = 1 to %elem(products2);
SortA products2(i).salesByMonth(*).sales;
EndFor;
// Once individual entries have been sorted into sales value
//
sequence sort the whole array into product code sequence
SortA products2(*).productCode;
Notes
Even nested arrays can be sorted as shown in the example below. In this case the inner array (i.e. the monthly sales
values) is sorted into ascending sequence and then the outer array (i.e. the products) are sorted into product code
sequence.
Of course, it doesn’t really matter which version of SORTA statement is done first - i.e., we could have sorted by
ProductCode first, followed by the loop to sort the sales figures.
Note, however, that there is still no support to sort on 2 (or more) different subfields in a DS array, unless the 2
subfields are contiguous in the DS and in the “correct” sequence.
© Partner400, 2015
Searching DS Arrays
%LOOKUP can now also search DS arrays
• The same asterisk (*) notation is used to indicate the search level
Only the vanilla %LookUp is supported
• Not the %LookUpGt etc. versions
D productInfo
D
D productCode
D description
D unitPrice
D qtyInStock
DS
D element
S
5a
40a
5p 2
5p 0
Dim(1000)
Qualified
5i 0
element = %LookUp( 'A123C': productInfo(*).productCode);
// Sort into price sequence and then find first value above $50.
SortA productInfo(*).unitPrice;
element = %LookUp( 50.00: productInfo(*).unitPrice);
Notes
To be truly useful, any enhancement in sorting needs to be matched with corresponding advances in searching, and
the RPG developers haven't let us down. They have enhanced the %LOOKUP BIF to allow for searching within DS
arrays.
At this time only the "exact match" %LookUp is supported. Hopefully %LookUpGt and other members of the family will
be supported in future releases. In the meantime you will have to write your own routine to do this.
© Partner400, 2015
Using ALIAS Names
ALIAS names can be used in Externally-described data structures
• By using the ALIAS keyword on the DS definition
ALIAS names can be used for file fields
• Use the ALIAS keyword on the File specification
✦
Any LIKEREC or EXTNAME DS based on the file will use the ALIAS name
DDS for file CUSTFILE
A
A
A
A
R CUSTREC
CUSTNM
CUSTAD
ID
D custDs
D
25A
25A
10P 0
e ds
ALIAS(CUSTOMER_NAME)
ALIAS(CUSTOMER_ADDRESS)
ALIAS QUALIFIED
EXTNAME(custFile)
custDs.customer_name = 'John Smith';
custDs.customer_address = '123 Mockingbird Lane';
custDs.id = 12345;
Notes
For many, many years going all the way back to the System/38, the database has supported the use of longer alias
names as an alternative to the cryptic 10 character names that we normally use. Indeed many COBOL shops have
always taken advantage of them. Usage of alias names has also increased in recent years with the growth in popularity
of SQL. But during all this time RPGers were locked out of using these longer names as the language was tied to the
old I and O specs and their limited field names.
When result field I/O was first introduced for externally described files - back in the V5R2 timeframe - we felt that this
might herald the arrival of Alias names into RPG. Well it has taken a few releases, but it is finally here. As from V7.1
you can specify the ALIAS keyword when defining an externally described DS, or any DS defined with LIKEREC. When
the ALIAS keyword is used, the compiler uses the longer alias name for the field rather than the short name. In cases
where the alias name does not meet RPG naming standards, the compiler reverts to using the short name.
Note that you put the ALIAS keyword on the F spec if you choose to create the DS using LIKEREC. However, if you
create the DS using EXTNAME, then use specify ALIAS on the DS description on the D spec.
© Partner400, 2015
Scan and Replace
New built-in function %SCANRPL
%SCANRPL ( scanFor : replaceWith : targetString )
• Replaces all occurrences of scanFor within the target string with the
contents of replaceWith
✦
Optional 4th & 5th parameters for scan-start-position and scan-length
✦
%SCANRPL( scanFor : replaceWith : target { : scan start { : scan length } )
Much simpler than having to code it the old way
• i.e. a %SCAN and %REPLACE loop
✦
Or %Scan and %Subst or ...
string1 = 'See &NAME. See
&NAME
run. Run
&NAME
run.';
string2 = %ScanRpl('NAME' : 'Forrest' : string1);
// string2 now contains 'See Forrest.
string3 = %ScanRpl('
See
Forrest run.
Run Forrest run.'
' : ' ' : string2); // Change double spaces to single
// string3 now contains 'See Forrest. See Forrest run. Run Forrest run.'
Notes
Note that in the example shown, replacing double spaces with a single space will only work on sets of double spaces
on the first pass through the string. In other words, if there were 4 spaces in a row, the new string would still have 2
spaces. If there were 3 spaces initially there would still be 2 spaces after the replacement.
But %ScanRpl can be used with a null string as the replacement - does that help? Well it does but it creates the
problem that if there were originally an even number of spaces then after the replacement there are none! Since we
don't know exactly where the spaces originally were, we have no idea where to put the required space back in.
Sometimes it seems you just can't win. But we have shown a possible solution on the next page - it looks odd - but it
works.
© Partner400, 2015
Scan and Replace - Example
The code below will remove all instances of multiple spaces
• And replace them with a single space
DSPLY op-codes allow you to see it happening step by step
d testString
d result
s
s
40a
40a
Inz('4 spc
Varying
7 spc
10+ spc ')
/free
result = %ScanRpl( ' ': '<>': testString ); // replace each space with <>
dsply ('Contents: ' + result );
result = %ScanRpl( '><': '': result ); // replace all >< with nothing
result = %ScanRpl( '<>': ' ': result ); // replace remaining <> with space
dsply ('Length of result is ' + %Char(%Len(result)));
Notes
This approach to the problem uses a rather strange looking sequence. It really does look "odd". For that reason I have
included a number of displays so that you can see the change in the string as each phase progresses. I suspect there
may be a better/shorter way of doing this but this one is fun anyway. If you decide to use it in a program PLEASE wrap
it up in subprocedure or you will confuse the heck out of those who follow you!
Below you can see the displays produced:
DSPLY
Contents: 4<>spc<><><><>5<>spc<><><><><>6<>spc<>
DSPLY
Contents: 4<>spc<>5<>spc<>6<>spc<>
DSPLY
Contents: 4 spc 5 spc 6 spc
DSPLY
Length of result is 18
© Partner400, 2015
Processing Result Sets in RPG
We could always write an RPG stored procedure to return a result set
• But we could not receive/process that result set in an RPG program
This new support may feel a bit "clunky" - But it works - You need to ...
• Define a RESULT_SET_LOCATOR (defined on D spec)
• Then ASSOCIATE the Result Set Locator with the Procedure
• Then ALLOCATE the CURSOR for the result set
D CustResultSet
S
SQLType(RESULT_SET_LOCATOR)
Exec SQL
Call CustomersByState( :InputState );
Exec SQL
Associate Result Set Locator (:CustResultSet)
with Procedure CustomersByState;
Exec SQL
Allocate C1 Cursor for Result Set :CustResultSet;
Exec SQL
Fetch next from C1 into :CustData;
Notes
If this syntax seems convoluted to you, we agree. It seems that it shouldn't need to be this complicated.
But at least we can now receive and process result sets from a stored procedure in an RPG program. It was always
incongruous that we could create the stored procedures that produced the result sets in RPG, but could not actually
use those same stored procedures from an RPG program if we chose to.
Note that once you associate the result set locator and allocate the result set to a cursor, you can then simply fetch
from that cursor as if it were a “normal” SQL cursor.
© Partner400, 2015
Relaxed Prototype Requirements
Prior to 7.1, the compiler requires a Prototype when:
• A subprocedure is compiled (even if called from a separate module)
• A subprocedure is called (even if the subprocedure is in same module)
In 7.1+, Prototypes are optional in some specific cases
• If the program or procedure is not called by another RPG module
✦
i.e. If not called from RPG, then the prototype will never be used
• Examples of optional prototypes include:
A program that is only intended to be used as an exit program or as the commandprocessing program for a command
✦ A program that is only intended to be called from a different (non-RPG) language
✦ A procedure that is not exported from the module (not callable externally)
✦ A procedure that is exported from the module but only intended to be called from a
different programming language
✦
• In other words, if the compiler can get the parameter list from a PI in the
same source member, the PR is optional
However, if you put the PR and the PI in the same source member, the
compiler will validate the PR to ensure it is correct - Bonus!
Notes
The RPG compiler required prior to V7.1 that you have a PR in every source member where a PI for that procedure or
program existed. The reason for this requirement was so that the compiler could validate that your PR was correct i.e., that it matched your PI. This is good because if the PR matches, then all calls to that program or procedure will be
validated by the compiler and parameter mismatches will be a thing of the past.
However, it was a bit painful that we had to put PRs in for cases where the PR would never be used by another RPGLE
program - it was only called by CL or some other language that couldn’t use our prototype - likewise for internal
subprocedures since the compiler could clearly get the parameter information directly from the PI, making the PR seem
redundant.
You should still include the prototypes in any source members where external subprocedures are coded so that the
compiler can check them for you. Use /Copy members for that purpose so that you can be sure that the version that
was validated when compiling the subprocedures is the same one used by all the callers of those routines.
© Partner400, 2015
New RTNPARM Prototype Keyword
Performance Boost for large return values
• Converts the return value to a hidden parameter
• Value reported by %PARMS( ) reflects the extra parameter
✦
✦
But it will otherwise be "invisible" to RPG callers
If calling from another language you will see the return value as the 1st parm
• Will improve performance when returning large values
✦
Especially very large varying values
Has a second unintended use
• It allows subprocs with return values to be used by Java or stored
procedure
✦
Prior to this change you were limited to a 4 byte integer (10i)
D getFileData
pr
a
Varying Len(1000000)
D
RtnParm
D
File
a
Const Varying Len(500)
D Data
S
a
Varying Len(1000)
/Free
Data = getFileData ('/home/mydir/myfile.txt');
Better Support for Optional Parameters
In the past, checked %Parms for a hard-coded value
• i.e., If %Parms > 1;
Now use %ParmNum( )
• Replaces hard-coded value for parm position
• %ParmNum returns the sequence of the parameter in the parm list
In example below, %ParmNum(Optional2) returns 2
• i.e., Optional2 is the 2nd parameter declared in the PI
D OptionalTest
D
Parm1
D
Optional2
PI
D Parm2
S
20A
5P 0 Options(*NoPass)
5P 0 Inz(99999)
// Check for optional parameter and use value if present
If %ParmNum(Optional2) <= %Parms;
Parm2 = Optional2;
EndIf;
If Parm2 .............
© Partner400, 2015
V7 - TR7 Free Form Enhancements
You can now specify an entire program in free form
! Only exception is that I and O specs are not available in free-form
No longer any need for /Free or /End-Free
! Just leave columns 6 and 7 blank
New free form options for:
! H-specs (CTL-OPT)
! F-specs (DCL-F)
! D-specs (DCL-xx)
- Where xx = C, DS, PARM**, PI, PR, S, or SUBF**
• ** These are rarely going to be used
! P-specs (DCL-PROC)
ctl-opt option(*srcstmt) dftactgrp(*No);
Not Technically a part of TR7 but
was announced and released at
the same time
dcl-ds employeeDS;
firstName char(16)
Inz('James');
lastname char(30)
Inz('Joyce');
salary
packed(7:2) Inz(12500);
end-ds;
// Define printer file and associated DS
dcl-f qprint printer(80);
dcl-ds prtDs len(80) end-ds;
Notes
Completely free form logic brings RPG more in line with other modern programming languages, all of which use free
format. This is important for attracting new developers coming into the marketplace.
Traditional fixed format RPG is far less attractive and gives RPG the undeserved appearance of an old-fashioned
language not up to modern application tasks. RPG is a powerful and flexible language that many young developers
come to prefer over other more popular language options for business applications.
But they must first be attracted to learn the language.
© Partner400, 2015
Format of the new Declarations
All of the new declaration op-codes follow this basic format:
! First the DCL-xx itself
! Next the name of the item
- File, field, procedure, etc.
! Followed by keywords related to other fixed form elements
- e.g. File usage, field type and length
! Then keywords from the old spec
We will be looking at a more
complete code sample on the
next chart
dcl-ds employeeDS;
firstName char(16) Inz('James');
lastname char(30) Inz('Joyce');
salary
packed(7:2) Inz(12500);
end-ds;
....
From original D-spec
keyword area
Fixed column
entries
end-ds can be on the same line
as dcl-ds in this case.
Notes
© Partner400, 2015
Simple Free Form Program
File definitions can be intermixed with data definitions
! Named Constants, Data Structures, etc.
Most new options have sensible defaults
! Disk files default to input, Printers to output, Decimals to zero, etc. etc.
End of line comments are now useful in definitions!
dcl-ds employeeDS; // Nice to be able to have comments here!
firstName char(16) Inz('James');
lastname char(30) Inz('Joyce');
salary
packed(7:2) Inz(12500);
end-ds;
// Define printer file and associated DS
// This printer is program described
dsply ('Hello to our new employee');
dsply ( %TrimR(firstName) + ' ' + lastName );
Note that the Printer is
defined together with the
DS that it uses for output
prtDs = 'The name of our new employee is ' +
%TrimR(firstName) + ' ' + %TrimR(lastName) +
' his salary is $' + %Char(salary);
write qprint prtds;
Notes
The idea of mixing file and data definitions will take some getting used to - but it makes sense.
After all it makes far more sense to define a set of variables, data structures and constants together with the file that
they will be used with that to arbitrarily separate them as we had to before.
© Partner400, 2015
CTL-OPT - The New H-Spec
Defaults to DftActGrp(*No) if any ILE specific options are used
! i.e. ACTGRP, BNDDIR, or STGMDL
- This is the first of many "sensible" defaults that are added by this support
Only other differences are:
! Can start in any column from 8 onwards
! Must be terminated with a semi-colon
! Presence of CTL-OPT stops compiler from looking for H spec data area
No other differences between this and the current H-spec
! Can occupy multiple lines
! Multiple CTL-OPT can appear in program
H BndDir('UTILBNDDIR') Option(*SRCSTMT: *NODEBUGIO)
Ctl-Opt BndDir('UTILBNDDIR');
Ctl-Opt Option(*SRCSTMT: *NODEBUGIO);
// Can also be coded as:
Ctl-Opt BndDir('UTILBNDDIR') Option(*SRCSTMT: *NODEBUGIO);
Notes
Not that much has changed about the H spec since it was almost free format before.
Ctl-Opt replaces the H and the semi-colon is used at the end.
© Partner400, 2015
Declaring Files In Free-Form
File Name listed first - followed by device type keyword (if any)
! Device type defaults to DISK - i.e., a Database table
Externally described is the default
! File Keyword *EXT can optionally be specified as a parameter
Program described files must specify their record length
! e.g. PRINTER(132) for a program described printer file
Defaults for USAGE are based on device type - more in a moment
! *Input, *Output, *Update (implies *Input), *Delete (implies *Update)
Add KEYED keyword for keyed database (disk) files
FCUSTMR0
UF A E
K DISK
FREPORT
O
E
PRINTER OFLIND(*IN96)
FSCREEN
CF
E
WORKSTN
Dcl-F CUSTMR0 DISK
USROPN
Usage(*Update:*Delete:*Output) Keyed UsrOpn;
Dcl-F REPORT
PRINTER(*EXT) OFLIND(*IN96);
Dcl-F SCREEN
WORKSTN Usage(*Input:*Output);
Notes
In the example here, we have defined 3 files - one is a keyed database table (aka file), a report, and an interactive
screen.
This example does specify some features that are not required. For example, PRINTER(*EXT) defining an externally
described printer did not need the *EXT parameter since externally defined is always the default. Also the
USAGE(*Input:*Output) on the Screen file is the default usage value for a display file and could have been omitted.
Note that Usage(*Output) is implied by default for the Report file.
The OFLIND keyword on the Report file is “Overflow Indicator” which can be used by the program to determine when to
go to a new page on the report.
There are other keywords that can be used when declaring files. These are the most commonly used features of file
declarations.
The File name is no longer limited to 10 characters as it was on the F spec. However, since file object names
(externally on the system) are limited to 10 characters, that means if a longer name is used, then keyword ExtDesc
must be coded to give the real external name for the file. A longer, more meaningful name for the file may be useful for
making the logic more readable.
© Partner400, 2015
DCL-F - Helpful Defaults
File name no longer limited to 10 characters
! So meaningful file names can be used
- EXTDESC is used to specify actual name when different from file name
Device type comes next followed by optional parameters
! Device type can be omitted if using an externally described Disk file
- A sensible default
All defaults are based on device type
! Usage(*Input) for DISK
! Usage(*Output) for PRINTER
! Usage(*Input : *Output) for WORKSTN
DCL-F InvoiceMaster ExtDesc('INVMAST');
// Defaults to Input Disk
DCL-F CustMaster
// Keyed Disk file
DCL-F qPrint
Usage(*Update) Keyed;
Printer(132) OflInd(PageFull); // Program described
DCL-F MyDisplay
WorkStn;
// Workstation Usage(*Input : *Output)
Notes
© Partner400, 2015
DCL-xx - The New D-Spec
xx = DS for Data structures
! In most cases there must also be a matching END-DS
xx = SUBF for DS subfields - Very Rarely Required
! Code only if field name is a valid free-form op-code
- Yes some strange people do use names like READ or SELECT as field names
xx = S for Stand-Alone fields
xx = C for Named Constants
D Address
D Street1
D City
D State
D Zip
D ZipPlus
DS
30A
30
2
5S 0
4S 0
Dim(20) Qualified
dcl-ds Address Dim(20) Qualified;
Street1
char(30);
City
char(30);
State
char(2);
Zip
zoned(5); // Zero decimals assumed
ZipPlus
zoned(4:0);
end-ds Address;
DS Name optional at end
Must match DS name
Notes
END-DS may be omitted if DS is externally described or has no named subfields. It can even be on the same line as
the DCL-DS in such cases, such as the following DS which is externally described based on a table (aka file) named
PRODUCT.
dcl-ds product
Ext end-ds;
Name of DS can be specified on end-ds and must match if present.
© Partner400, 2015
DCL-xx - The New D-Spec - Data Types
Data types are spelled out instead of using a one character code
! Chart below shows the most commonly used types
! Length or format of item follows in parentheses when required
Simpler definition for some data types
! VarChar avoids the need for the Varying keyword
! Date, Time, Timestamp don't need separate DatFmt keyword
! 0 (zero) decimals assumed for all numeric data types if not specified.
Data Type
Free Keyword
Notes
A
A+
Varying
I
U
P
S
N
D+
DatFmt
B
Char(len)
Varchar(len)
Int(len)
Uns(len)
Packed(len:dec)
Zoned (len:dec)
Ind
Date(format)
BinDec
(len:dec)
Decimals not specified
Decimals not specified
0 assumed if decimals not given
0 assumed if decimals not given
!! DO NOT USE !!
The following is a partial list of RPG data types represented as D-specs:
d packedNum
s
d zonedNum
s
7p 2
7s 2
d integer
s
10i 0
d unsigned
s
10u 0
d float
s
8f
d character
s
20a
d varyingChar
s
20a
d dateMDY
s
d
DatFmt(*MDY)
d timeUSA
s
t
TimFmt(*USA)
d indicator
s
n
d nastybinary
s
9b 0
Varying
And here are their free form equivalents.
Dcl-S packedNum
Packed(7:2);
Dcl-S zonedNum
Zoned(7:2);
Dcl-S integer
Int(10);
Dcl-S unsigned
Uns(10);
Dcl-S float
Float(8);
Dcl-S character
Char(20);
Dcl-S varyingChar
Varchar(20);
Dcl-S dateMDY
Date(*MDY);
Dcl-S timeUSA
Time(*USA);
Dcl-S indicator
Ind;
Dcl-S nastybinary
Bindec(9);
© Partner400, 2015
D-Spec DS Example
This shows how a fixed form DS would be converted to free-form
• I have used my own personal preferences for alignment
✦
At least they were my preferences at the time I coded this example!
D MyDS
D Address
D
Street
D
City
D
State
D
Zip
D LstOrdDate
D ASubfield
Dcl-DS MyDs;
Address
Street
City
State
Zip
LstOrdDate
ASubfield
End-DS;
DS
32A
15A
10A
2A
5A
D
25A
Overlay(Address)
Overlay(Address: *Next)
DatFmt(*USA)
Varying
Char(32);
Char(15) Overlay(Address);
Char(10) Overlay(Address:16);
Date(*USA);
Varchar(25);
Notes
I like the idea of aligning the field type definitions even though it is not required. I don't however insist on placing them
in a specific column. Rather I start them 2 characters after the end of the longest field name in the block. For me it
works but you'll devise your own style - just be consistent and remember that readability is critical.
© Partner400, 2015
D-Spec - Unnamed Fields and Overlays
In fixed form no field name was required
• In free-form we must specifically inform the compiler
✦
Done by coding *N instead of a field name
In free-form we are not permitted to OVERLAY the DS name
• IBM decided that it should never have been allowed so they stopped it
• So we must use POS(1) instead
✦
POS(nn) can also be used for positioning any field within a DS
✦
For example when specifying specific fields in the PSDS
Dcl-DS DayData;
*n
Char(9)
*n
Char(9)
*n
Char(9)
*n
Char(9)
*n
Char(9)
*n
Char(9)
*n
Char(9)
DayArray
inz('Monday');
inz('Tuesday');
inz('Wednesday');
inz('Thursday');
inz('Friday');
inz('Saturday');
inz('Sunday');
Char(9) Dim(7) Pos(1);
End-ds;
Notes
The RPG compiler team decided that too many people got confused when the OVERLAY keyword was used against
the DS name. To help avoid this confusion, OVERLAY is now limited to subfields within the DS and you must use the
POS keyword to reference the DS starting position.
This is effectively the same as using a start position on the old D-specs.
This is what the DS would have looked like in fixed-form.
D DayData
DS
D
9
Inz('Monday')
D
9
Inz('Tuesday')
D
9
Inz('Wednesday')
D
9
Inz('Thursday')
D
9
Inz('Friday')
D
9
Inz('Saturday')
D
9
Inz('Sunday')
D DayArray
9
Overlay(DayData) Dim(7)
© Partner400, 2015
More on Data Definition
Constants can be used in many more places
• Including field length, decimal places, array dimensions - just about
anywhere you would use a literal
Long names don’t require ellipsis … and a continuation line
D DateMDY
S
D
D array
S
10a
dim(10)
50A
Varying
D CustomerInfo
DS
D
CustomerName
D
CustomerBalance...
D
DatFmt(*MDY-)
7P 2
dcl-s DateMDY
Date(*MDY-);
dcl-S array
Char(10) Dim(10);
dcl-c
dcl-c
Digits 7;
Decimals 2;
dcl-ds CustomerInfo;
CustomerName
CustomerBalance
end-ds;
VarChar(50);
Packed( Digits: Decimals);
Notes
Constants can be used for length definition. Makes it really easy to change all currency fields from say 7,2 to 9,2 !!!
Note that in the fixed format example here, it made no sense to define Digits and Decimals as constants since they
could not be used to define the length and precision in fixed format D specs. However, in the new free format D specs,
they can be used - as illustrated in the 2nd example.
Note that not only are the ellipsis not required for longer variable names - they are not allowed in most cases. Unless
the actual variable name is continued on the next line, ellipsis should not be used, even if the data type and other
related keywords are on the next line. If ellipsis are found, the compiler assumes the next line will contain part of the
variable name.
© Partner400, 2015
Data Areas
This is an area where you may trip up
• D specs only allowed literals - constants were not allowed
✦
So quotation marks were not compulsory
• Now you will probably get a compiler error if you forget the quotes
✦
It will be looking for a constant named JonsData
• And of course you must specify the name all in upper case
D myDataArea
D
lastRunDate
D
lastSeqNum
D
lastInvNum
ds
Dcl-Ds myDataArea
d
5i 0
7p 0
DtaAra(JonsData);
DtaAra(JonsData)
DatFmt(*USA)
// Wrong unless
// JonsData is a constant
Dcl-Ds myDataArea DtaAra('JONSDATA');
lastRunDate Date(*USA);
lastSeqNum
Int(5);
lastInvNum
Packed(7);
End-Ds;
Notes
This is one of a number of similar areas where previously the compiler only allowed a literal - so even if quotes were
not used the compiler just assumed them and was quite happy.
This new support though allows for far more widespread use of constants and subsequently where literals are used
they must _be_ literals - complete with quotes and the data in the right case.
With great power comes ...
© Partner400, 2015
Subprocedures, Prototypes and More
DCL-PROC declares a subprocedure
• Completed by an END-PROC
DCL-PI and DCL-PR define procedure interfaces and prototypes
• Placeholder *N can be used if you don't want to type the name again
• DCL-PARM is the optional equivalent of DCL-SUBF within PIs and PRs
✦
Only needed if name of parameter is the same as an RPG op-code
New option for EXTPROC - *DCLCASE
• Means the real name is exactly the same as the procedure or prototype name
✦
Avoids having to retype the name when using mixed case procedure names
dcl-proc DayOfWeek Export;
dcl-pi *N Int(3) ExtProc(*DclCase);
InputDate Date(*USA) Value;
end-pi;
// Omit name - use *N placeholder
dcl-s DayNumber int(3);
// Do calcs leaving value in DayNumber
Return DayNumber;
end-proc DayOfWeek;
Notes
The biggest advantage of the new support is that you no longer have to flip in and out of fixed and free modes when
coding subprocedures. No more /End-Free, P-specs, D-Specs, /Free, logic, /End-Free etc. It makes it all look much
cleaner and removes a major source of mistakes (and frustration) for those learning RPG.
© Partner400, 2015
Subprocedures, Prototypes and More
EXTPGM keyword can be omitted
• Providing that the program is non-ILE i.e. DFTACTGRP(*YES)
Program name can be omitted from EXTPGM
• If the program name is the same as the prototype
EXTPGM('PROGNAME')
• Parameter only needed when proto name is different from actual program name
dcl-pr MyProgram; // Used to call 'MYPROGRAM' from non-ILE program
dcl-pr MyProgram ExtPgm; // Calls 'MYPROGRAM' from any program
dcl-pr DifferentName Extpgm('MYPROGRAM'); // Call 'MYPROGRAM using the
//
name DifferentName
Notes
Once again the new support adds some sensible and useful defaults.
The more I use this stuff the more I love it!
© Partner400, 2015
Converting to Fully Free Form
IBM does NOT supply a tool to do this
! Even RDi's free-form converter has not been updated
Currently we know of two tools that are available
! Linoma's RPG Toolbox
- Which also does a great job of converting old-style fixed-form to free form
• Details at linomasoftware.com/products/rpgtoolbox/
- Quick demonstration in a moment
! Arcad Transformer
- A far more all-embracing modernization tool - but more expensive
• Details at arcadsoftware.com/products/arcad-transformer-ibm-i-refactoring-tools/
- It can even refactor code to the extent of getting rid of GOTOs and other similar
ancient constructs.
Notes
We have been using Linoma's conversion tool since way back in the V3 days when RPG IV was first introduced. Over
the years they have steadily added more and more functionality including the ability to apply "aggressive" conversion
options that deal with converting various flavours of MOVE operations.
With the latest updates they also introduced a low-cost add on to the toolbox that works as a plug-in to RDi and does a
great job of converting whole prong ams or just selected pieces.
Another nice feature, not related to free-form, is an indenting option that re-establishes correct indentation after you
have changed the structure of an If, For, Dox, etc. block.
Arcad's tooling is more recent but much more all encompassing. I like what I have seen of it but have not had any
long-term experience with the tool.
© Partner400, 2015
Linoma's RPG ToolBox - Side By Side Comparison
Linoma's tool does a great job but ...
! It tends to play it safe and includes a lot of extra keywords
! I'm also not a fan of the way it aligns things - it doesn't!
- But they are looking at changing this and providing more formatting options
After
Before
Notes
My biggest grouse with the Linoma tool is that it converts everything and that means it also adds a bunch of file
declaration keywords that are not necessary. The first thing I do after running a conversion with the tool is to manually
clean it up.
Their conversion of D-specs does not apply any alignment to the resulting code - just inserts a single blank between
the components. For me the result looks messy so again I apply some manual clean up after conversion. An example
on the next chart.
© Partner400, 2015
Edited Version of Converted Code
I removed the superfluous file entries and applied my own alignment
style to the data declarations.
Notes
Here you can see that I have removed the superfluous keywords from the file declaration. I have also added some
alignment to the data declarations.
Right now my basic alignment rule is that in any given group (i.e. DS, PR, PI, series of standalone fields, etc.) I insert
two spaces between the longest data name in the group and its data type/length definition and then align all other
entries in the group to that. You can see the effect in the chart.
Note that I don't try and make all the entries in all the groups align - that is just too much work and I don't find it
necessary - but visually I do like the entries in a specific group to align.
Choose your own style - but try to make sure that everyone in the shop uses the same (or very similar) style.
© Partner400, 2015
Time to Update Your Editor
This is what free-form coding looks like in SEU
! IBM will NOT be enhancing SEU to handle these new definitions
Time you moved up to a real editor - RDi
Notes
IBM has confirmed that SEU will not be updated to support this new functionality. In fact it was frozen as at V6 and
supports none of the V7 functionality.
As a result a great many people have started doing what they should have done many years ago and moved their
development to RDi. It fully supports the new free-form including excellent code-assist to help you remember the new
formats.
© Partner400, 2015
"Gotchas" to Watch Out For
OVERLAY keyword cannot be used against DS name
! Use POS(n) instead
Names in EXTNAME, EXTFLD, and DTAARA
! Must be in quotes and are case sensitive
! Without quotes, they are treated as a variable or constant name
Ellipsis (...) for continuation only allowed when continuing a name
! But not really needed anymore anyway
On F-Spec "U" enables update and delete
! In free form *DELETE must be requested explicitly
End-DS, End-PR, End-PI are always required
! But may appear on same line as DCL-xx in some cases
RDi's "Convert all to Free-Form" means only convert "all logic"
! And will still generate /Free and /End-Free
I and O specs remain in fixed form
! Probably forever
Notes
© Partner400, 2015
The High Points
Intelligent defaults
! For device type (Defaults to DISK)
! Usage (Default based on device type)
! Decimal places (Defaults to zero)
Constants can be used as keyword values
! Including for lengths and decimal places
DFTACTGRP(*NO) is optional
! If any other ILE keyword such as ACTGRP or BNDDIR is used
File and data declarations can be mixed
! Even in fixed form
/Free and /End-Free no longer required
/Copy and other compiler directives no longer need to start in col 7
// style end of line comments can be used in data/file declarations
Notes
© Partner400, 2015
Resources
Linoma Software's RPG Toolbox
• The latest evolution of the first RPG III conversion utility
✦
Accommodates Free-form formatting and much more
• www.LinomaSoftware.com and follow the RPG Toolbox links
Articles by Jon Paris and Susan Gantner
• Search for them from the IBM Systems Magazine site
✦
www.ibmsystemsmag.com/authors/Susan-Gantner/
✦
www.ibmsystemsmag.com/authors/Jon-Paris/
Free-Format RPG IV: Third Edition
✦
✦
✦
by Jim Martin
Published by MCPress (www.MC-store.com)
Make sure to order the third edition - older versions do NOT contain the V7 free-form
additions
Notes
As noted on the chart - if buying Jim Martin's book make sure you get the third edition or later. The earlier versions do
not include the V7 enhancements that we have been discussing in this session.
© Partner400, 2015
Advanced Data Structures
Now with new Free
Form Definitions!!
Jon Paris
www.Partner400.com
Agenda
What do I mean by "Advanced" ?
DS stuff that you might not know
No-length fields
No-name fields
Incorporating external fields into a DS
Group fields
Mapping Indicators to Names
Data Structures
V5R2 - LikeDS and LikeRec
Templates
Subfile Sort Example
Using many of these capabilities
© Partner400, 2015
Unnamed Fields and Overlaying a DS
DS subfields do not need to be named
• But they can still have INZ values!
Makes a great alternative to using compile-time data
• Initialize the data near the array definition itself
• No need to chase to the end of the source member
• The fields can have names if you wish but they do not have to
Our passionate dislike of compile-time data led us to this technique
• It makes life much simpler when you want to see the values used to
intialize an array.
D Messages
D
D
D
D
D
DS
Msg
20a
20a
20a
20a
Inz('Invalid Item Code')
Inz('Too many selections')
Inz('Item Code required')
Inz('Huh?')
20a
Overlay(Messages) Dim(4)
Unnamed Fields and Overlaying a DS
DS subfields do not need to be named!
• You can use the placeholder *N instead
• And they can still have INZ values!
In Fixed form the Overlay keyword can apply to the DS
• In free-form you must use the POS keyword to position the redefinition
Makes a great alternative to using compile-time data
• Initialize the data near the array definition itself
• No need to chase to the end of the source member
dcl-ds Messages;
*n
Char(20)
*n
Char(20)
*n
Char(20)
*n
Char(20)
Msg
end-ds;
© Partner400, 2015
Free Form
Version
Inz('Invalid Item Code');
Inz('Too many selections');
Inz('Item Code required');
Inz('Huh?');
Char(20) Dim(4) Pos(1);
No-Length Subfields
Want to access the fields in a record as an array?
The "secret" is to place them in a DS
This technique works even if the fields are not contiguous in the record
No length definition is required
The compiler will use the length supplied by the database
Next overlay the array against the DS name
Redefining the fields as an array
Notice the use of the LIKE keyword
D SalesData
D Q1
D Q2
D Q3
D Q4
D SalesForQtr
D
DS
R MTHSALESR
CUSTOMER
STREET
CITY
STATE
DIVISION
Q1
Q2
Q3
Q4
K CUSTNO
4
32
24
2
2
7
7
7
7
DDS
2
2
2
2
Overlay(SalesData)
Like(Q1) Dim(4)
Notes
The inspiration for this example comes from one of the most commonly asked questions on RPG
programming lists: “How do I directly load fields from a database record into an array?” The question
normally arises when each database record contains a series of related values - for example, monthly
sales figures.
The DDS for the physical file is shown. One solution is depicted here. We'll look at a different solution
on the next chart. Our objective is to access the individual fields Q1-Q4 as an array of four elements
after reading a record from the file.
Notice that we’ve incorporated the Quarterly sales fields into the DS by specifying their names. No
length or type definition is required. Instead of allocating storage for the file’s fields arbitrarily, the
compiler is told to explicitly place the data in the DS. Because the DS no longer contains the Customer
and Division data, we can use the simple form of the Overlay keyword. Otherwise, if we had used an
externally described DS, we would have needed to place a starting position on the Overlay keyword.
The code for that alternative solution is shown below. Unlike the externally described DS, The
example on this chart allows you to see exactly which fields form the DS. In addition, the fields in the
DS can come from multiple files.
D SalesData
E DS
ExtName(TestFile)
// Define array over Quarterly Sales figures
D SalesForQtr
Overlay(SalesData: 7)
D
Like(Q1) Dim(4)
© Partner400, 2015
No-Length Subfields
Want to access the fields in a record as an array?
The "secret" is to place them in a DS
This technique works even if the fields are not contiguous in the record
No length definition is required
The compiler will use the length supplied by the database
Use the POS keyword to cause the array to overlay the DS
Notice the use of the LIKE keyword
dcl-f MthSales;
dcl-ds
SalesData;
Q1;
Q2;
Q3;
Q4;
SalesForQtr
end-ds;
Free Form
Version
Like(Q1)
Pos(1)
R MTHSALESR
CUSTOMER
STREET
CITY
STATE
DIVISION
Q1
Q2
Q3
Q4
K CUSTNO
4
32
24
2
2
7
7
7
7
DDS
2
2
2
2
Dim(4);
Notes
© Partner400, 2015
Group Fields
Sometimes it is convenient to reference a group of fields
You might wish to group Street, City and State under the name Address.
That way they can be manipulated as a single entity
This version of the previous example uses this approach
The compiler derives the length of the group field from the combined
lengths of the subfields that OVERLAY it
D SalesData
D Customer
D Address
D
Street
D
City
D
State
D Division
D QuarterData
D
Q1
D
Q2
D
Q3
D
Q4
D
DS
Overlay(Address)
Overlay(QuarterData)
Overlay(QuarterData: *Next)
SalesForQtr
Overlay(QuarterData) Like(Q1) Dim(4)
Notes
Note that neither Address or QuarterData have a length, type definition or LIKE keyword. Nor do they
exist in any of the program’s files. Normally you’d expect such definitions to result in Field Not Defined
errors, but it doesn’t because the subsequent OVERLAY references inform the compiler that these
fields represent a group field.
If you look in detail at QuaterData, you will see that it comprises the fields Q1 though Q4. If you
examine the extract from the compiler cross-reference listing below, you’ll see that QuarterData is
defined as 28 characters long (i.e., the combined lengths of Q1, Q2, Q3 and Q4):
Q1
Q2
Q3
Q4
QUARTERDATA
SALESDATA
SALESFORQTR(4)
© Partner400, 2015
S(7,2)
S(7,2)
S(7,2)
S(7,2)
A(28)
DS(34)
S(7,2)
14D
15D
16D
17D
13D
10D
19D
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Group Fields
Sometimes it can be convenient to reference a group of fields
• e.g. To group Street, City and State under the name Address.
✦
That way they can be manipulated as a single entity
The compiler derives the length of the group field
• It is the sum of all fields that OVERLAY it
Free Form
Version
dcl-f MthSales;
dcl-ds SalesData;
Customer;
Address;
Street
City
State
Division;
QuarterData;
Q1
Q2
Q3
Q4
SalesForQtr
end-ds;
Overlay(Address);
Overlay(Address: *Next);
Overlay(Address: *Next);
Overlay(QuarterData);
Overlay(QuarterData: *Next);
Overlay(QuarterData) Like(Q1) Dim(4);
Notes
© Partner400, 2015
Using SORTA with Group Fields
Want to sort an array on different keys?
Group fields can provide an answer
ProductData is a group field
It comprises the fields Name, UnitPrice and QtyInStock
Notice that the DIM is specified at the group level
This allows the array to be sorted on any of the subfields
The associated data will "follow along" and stay in sync
D ProductInfo
DS
// Note that Dim is specified at the group field level
D
ProductData
Dim(1000)
D
Name
20
Overlay(ProductData)
D
UnitPrice
7p 2 Overlay(ProductData: *Next)
D
QtyInStock
/Free
SortA Name;
// Sort into Name sequence
SortA UnitPrice;
// Sort in Unit Price sequence
Notes
Note that when using this technique all of the other fields in the array (i.e. those that are part of the
group) will be "pulled along" with their associated values.
ASCEND or DESCEND can be specified as normal along with the DIM keyword. So, while you can
sort on any of the fields in the group, you can only sort ascending OR descending sequence on any
given array.
In order to allow alternate sequencing you could use a pointer to base a second version of the array as
shown in the example below:
D ProductInfo
D
ProductData
D
Name
D
UnitPrice
D
QtyInStock
DS
Dim(1000) Acsend
8
Overlay(ProductData)
8
Overlay(ProductData: *Next)
// Use a Based version of the DS to allow the alternate sorting seq.
D AlternateView
D
ProductDataD
D
NameD
D
UnitPriceD
D
QtyInStock
DS
D pProductInfo
S
© Partner400, 2015
Based(pProductInfo)
Dim(1000) Descend
8
Overlay(ProductDataD)
8
Overlay(ProductDataD: *Next)
9p 0 Overlay(ProductDataD: *Next)
*
Inz(%Addr(ProductInfo))
Using SORTA with Group Fields
Want to sort an array on different keys?
• Group fields can provide an answer
ProductData is a group field
• It comprises the fields Name, UnitPrice and QtyInStock
Notice that the DIM is specified at the group level
• This allows the array to be sorted on any of the subfields
• The associated data will "follow along" and stay in sync
Free Form
Version
dcl-ds ProductInfo;
// Note that Dim is specified at the group field level
ProductData Dim(1000);
Name
char(20)
Overlay(ProductData);
UnitPrice
packed(7:2) Overlay(ProductData: *Next);
QtyInStock
packed(7:2) Overlay(ProductData: *Next);
end-ds;
SortA Name;
// Sort into Name sequence
SortA UnitPrice;
// Sort in Unit Price sequence
Notes
Note that when using this technique all of the other fields in the array (i.e. those that are part of the
group) will be "pulled along" with their associated values.
Prior to V7 if you needed to sort the array with SORTA you could only specify ASCEND or DESCEND
on the actual array definition to control the sequence. In V7 IBM added the ability to specify the
required sort sequence as an operation extender to SORTA. So SORTA(A) will sort the array in
ascending sequence and SORTA(D) in descending order.
© Partner400, 2015
Mapping The *INnn Indicators to Names
You can use a pointer to map the standard indicator array
• An easy way to give names to all of your indicators
The technique shown associates names with the *INnn indicators
• Adding _nn to the name allows you to document the actual indicator
number that is associated with the name
• Thanks to Aaron Bartell for introducing us to this idea
D DspInd
DS
// Response indicators
D
Exit_03
D
Return_12
D
D
D
// Conditioning indicators
Error_31
StDtErr_32
EndDtErr_33
D pIndicators
S
Based(pIndicators)
N
N
Overlay(DspInd: 3)
Overlay(DspInd: 12)
N
N
N
Overlay(DspInd: 31)
Overlay(DspInd: 32)
Overlay(DspInd: 33)
*
Inz(%Addr(*In))
Notes
Unlike the INDDS approach, these named indicators DO directly affect the content of their corresponding *IN
indicator. If we EVAL Error = *On, then indicator *IN30 was just turned on. This often makes this a better approach
for those who use program described (i.e. O-spec) based files rather than externally described printer files.
Those of you who use the *INKx series of indicators to identify function key usage need not feel left out. A similar
technique can be used. IN this case the pointer is set to the address of *INKA. The other 23 function key indicators
are in 23 bytes that follow. IBM have confirmed many times that for RPG IV this will always be the case.
D FunctionKeys
DS
Based(pKxIndicators)
D
thisIsKC
N
Overlay(FunctionKeys: 3)
D
thisISKL
N
Overlay(FunctionKeys: 12)
*
Inz(%Addr(*InKA))
D pKxIndicators
© Partner400, 2015
S
Mapping The *INnn Indicators to Names
You can use a pointer to map the standard indicator array
• An easy way to give names to all of your indicators
The technique shown associates names with the *INnn indicators
• Adding _nn to the name allows you to document the actual indicator
number that is associated with the name
• Thanks to Aaron Bartell for introducing us to this idea
Free Form
Version
dcl-ds DspInd
Based(pIndicators);
// Response indicators
Exit_03
Ind Pos(3);
Return_12
Ind Pos(12);
// Conditioning indicators
Error_31
Ind Pos(31);
StDateError_32
Ind Pos(32);
EndDateError_33
Ind Pos(33);
end-ds;
dcl-s pIndicators
The use of POS
makes it more
readable than
OVERLAY
Pointer Inz(%Addr(*In));
Notes
Unlike the INDDS approach, these named indicators DO directly affect the content of their corresponding *IN
indicator. If we EVAL Error = *On, then indicator *IN30 was just turned on. This often makes this a better approach
for those who use program described (i.e. O-spec) based files rather than externally described printer files.
Those of you who use the *INKx series of indicators to identify function key usage need not feel left out. A similar
technique can be used. In this case the pointer is set to the address of *INKA. The other 23 function key indicators
are in 23 bytes that follow. IBM have confirmed many times that for RPG IV this will always be the case.
dcl-ds FunctionKeys
Based(pFunctionKeys);
F3_Pressed
Ind Pos(3);
F12_Pressed
Ind Pos(12);
end-ds;
dcl-s pFunctionKeys
© Partner400, 2015
Pointer Inz(%Addr(*InKA));
Data Defintion Update
V5R2 and Beyond
Data Definition Features
These are essential to processing XML
with RPG's built-in XML-INTO support
Not all charts in this section include a
Free Form Version
(But hopefully you'll have got the idea by
now)
Notes
V5R2 saw the biggest change in data definition capabilities in the history of RPG. Many new language features are
based on these capabilities. For example, both XML processing and RPG Open Access rely on them. You NEED to
understand them.
© Partner400, 2015
Qualified Data Names & LIKEDS
DS Keyword LIKEDS
• The new DS inherits all the fields of the named DS
• This includes all field names and their size and data type attributes
✦
But NOT any DIM or OCCURS specifications on the original DS definition
• Use of LIKEDS implicitly adds the QUALIFIED keyword to the new DS
• Use INZ option *LIKEDS to clone initialization values
Avoids the need to use suffixes or prefixes
• You can now have multiple versions of the same field
D baseYear
S
4s 0
D date
D
year
D
month
D
day
DS
D orderDate
DS
4s 0 Inz(2003)
2s 0 Inz(01)
5s 0 Inz(01)
LikeDS(date) Inz(*LikeDS)
If year > baseYear;
...
If orderDate.year > baseYear;
...
Notes
Previously in RPG one field name = one storage location. As a result if a field name appeared in two different files
then reading either file would change the field's content. This often necessitated using prefixes or suffixes on the
base name to differentiate between the versions. You might have ARACCNO and CMACCNO for example, both of
which are actually the account number (ACCNO). Now that we can use qualified data names in RPG the need for
this diminishes significantly. It will take time to adjust the way we deal with files, but in our programs we can now
have multiple versions of a field name differentiated by the DS name.
The new keyword LIKEDS causes all fields and their definitions in the original DS to be cloned in the new DS. By
definition the new DS is automatically qualified.
In addition there is also a new parameter value allowed on the INZ keyword for data structures defined with the
LIKEDS keyword: INZ(*LIKEDS).
This means that any initial values specified in the original data structure should be replicated in the new (LIKEDS)
data structure as well. For example, if in our code sample on this chart had an initial value for the Year subfield in
Date, such as INZ(2001), that initial value would ONLY be carried over to the OrderDate.Year field IF we added the
keyword INZ(*LIKEDS) to the OrderDate D spec.
There is a particularly good discussion and example of this support as it relates to passing a Data Structure as a
prototyped parameter in the article by Hans Boldt and Barbara Morris in IBM's iSeries Magazine, May 2001 issue.
© Partner400, 2015
Qualified Data Names & LIKEDS - more
Free-form works in basically the same way as fixed
• Note that the Dim(10) on the original DS is not cloned
• In this example the parent DS was qualified
✦
It can help avoid confusion if all instances of a field name are qualified
• Note the positioning of the array indices
✦
✦
They belongs with the name of the item that had the Dim statement
More on DS arrays later
dcl-ds date
year
month
day
end-ds;
Dim(10) Qualified;
zoned(4) Inz(2003);
zoned(2) Inz(01);
zoned(5) Inz(01);
The DIM here will
NOT be cloned
- only the subfield
definitions
dcl-ds orderDate Dim(99) LikeDS(date) Inz(*LikeDS);
if date.year > baseYear;
...
if orderDate(1).month = orderDate(2).month;
...
Notes
The free form implementation is basically the same - just prettier!
In this chart we have also introduced the concept of Data Structure arrays. These are a major improvement on the
old Multiple Occurrence Data Structures (MODS) which are still supported for compatibility reasons. We will be going
into more detail on DS arrays in a few minutes.
Notice that even though the original array had a DIM statement, this is NOT cloned. The new DS must specify its
own DIM if it is to be an array.
© Partner400, 2015
Nested DS
The LIKEDS keyword is used to specify the DS to be nested
• Any DS that is to contain a nested DS(s) must specify QUALIFIED
To reference the fields requires double qualification
• e.g. InvoiceInfo.MailAddr.City or InvoiceInfo.ShipAddr.State
But wait - there's more !!
• Data Structure arrays !!!!
D Address
D
Street1
D
Street2
D
City
D
State
D
Zip
D
ZipPlus
DS
D InvoiceInfo
D
MailAddr
D
ShipAddr
DS
30a
30a
20a
2a
5s 0
4s 0
LikeDS(Address)
LikeDS(Address)
QUALIFIED
Notes
Note that when the LIKEDS keyword is used to reference a DS array, the DIM characteristic is not copied by the
LIKEDS. Only the individual components of the DS (fields, conventional arrays and nested DSs) are copied.
Look at this example:
D DSArray
DS
D Data
D NewDS
Dim(20) Qualified
12a
DS
D NewDSArray
Qualified
LikeDS(DSArray)
Even though DSArray is itself an array, when we reference it via the LIKEDS keyword in NewDS, the DIM
characteristic is not inherited. In order to establish NewDSArray as a array DS, we explicitly code the DIM keyword,
as shown below.
D NewDS
D NewDSArray
© Partner400, 2015
DS
Qualified
LikeDS(DSArray) Dim(20)
Data Structure Arrays
The DIM keyword can now also be specified at the DS level
• More useful than MODS since all levels are accessible at once
• Keyword QUALIFIED is also required
✦
But we're not quite sure why
Subscripting works in the same way as for individual fields
• Address(5) is the whole of the fifth element of the Address DS array
• Address(5).City is the City field within that fifth element
D Address
D
Street1
D
Street2
D
City
D
State
D
Zip
D
ZipPlus
DS
DIM(20) QUALIFIED
30a
30a
20a
2a
5a
4a
/Free
If Address(5).City = 'Rochester';
Address(5).State = 'MN';
EndIf;
Notes
In V6 a new keyword was introduced - TEMPLATE - to save on storage when we simply want to define our own data
types - i.e. DS and fields that are simply going to be used as templates for other DS and fields via LIKEDS and LIKE.
We will study that in a moment or two
© Partner400, 2015
Multidimensional Arrays
If your DS contains an array .....
• You now have the effect of a multidimensional array
The names are subscripted just as you'd expect them to be
• So - Address(5).Street(1) refers to the first element in the Street array
within the fifth element of the Address DS array
D Address
D
Street
D
City
D
State
D
Zip
D
ZipPlus
DS
30a
20a
2a
5a
4a
DIM(20) QUALIFIED
DIM(2)
/Free
Address(5).Street(1) = *Blanks;
:
:
:
If Address(5).Street(2) = *Blanks;
:
:
:
Notes
Although we can effectively create multidimensional arrays using this technique, things like SORTA and %LOOKUP
are limited in what they can do for us. This situation was not resolved until V7
© Partner400, 2015
Another Example
The DS array SalesByYear has ten elements
• Each of which contains a 12 element array and a total field
• Both sets of logic will sum up the elements in each Sales4Month array
and place the total in the corresponding Total4Year field
D SalesByYear
D
Sales4Month
D
Total4Year
DS
Dim(10) QUALIFIED
7p 2 Dim(12)
9p 2 Inz
/Free
For Y = 1 to %Elem(SalesByYear);
SalesByYear(Y).Total4Year = %XFoot(SalesByYear(Y).Sales4Month);
EndFor;
OR
For Y = 1 to %Elem(SalesByYear);
For M = 1 to %Elem(SalesByYear.Sales4Month)
SalesByYear(Y).Total4Year = SalesByYear(Y).Total4Year
+ SalesByYear(Y).Sales4Month(M)
EndFor
EndFor
Notes
I have included an example of manually looping through the inner array to build the totals because sometimes
%XFoot will not do the job.
Suppose that you want to create year-to-date values in the array, or quarterly totals. For that you would have to loop
thorough manually as shown in the second example.
© Partner400, 2015
Entering the Fourth Dimension
You can create as many dimensions as you like but ...
•
•
•
•
Until V6 a single element at the top level DS could not exceed 64K
In the example below changing Departments to DIM(11) broke the limit
See the notes page for the size calculations
Increasing Divisions to (say) DIM(99) has no impact
✦
Other than the 16Mb storage limit for data items
D Divisions
D
DivCode
D
Departments
DS
D DeptData
D
DeptCode
D
Products
DS
D ProductData
D
ProdCode
D
MonthsSales
DS
QUALIFIED Dim(5)
2s 0
LikeDS(DeptData) Dim(10)
QUALIFIED
3s 0
LikeDS(ProductData) Dim(99)
QUALIFIED
5s 0
9p 2 Dim(12)
/free
Divisions(1).Departments(1).Products(1).MonthsSales(1) = 0;
Notes - DS Size issues prior to V6
To see why the simple change of increasing the dimension of the Department DS array to 11 causes the storage requirement to
exceed the V5R4 size limit study the size of the nested DSs.
D ProductData
D
D
DS
ProdCode
MonthsSales
QUALIFIED
5s 0
9p 2 Dim(12)
ProductData is made up of the 5 byte Department code and a 12 element array. Each element is 5 Bytes long so the size of
ProductData is: 5 + (12 x 5) = 65 bytes.
D DeptData
D
DeptCode
D
Product
DS
QUALIFIED
3s 0
LikeDS(ProductData) Dim(99)
DeptData consists of the 3 byte Department Code and 99 instances of the ProductData DS Array. So the size is: 3 + (99 X 65) =
6,438 bytes.
D Divisions
D
DivCode
D
Department
DS
QUALIFIED Dim(5)
2s 0
LikeDS(DeptData) Dim(10)
Division consists of the 2 byte Division Code and 10 instances of the DeptData DS array. So its size is: 2 + (10 X 6,438) = 64,382.
This is under the V5R4 limit of 65,535, but not by much. Simply increasing the DIM on the Department DS array to 11 raises the
requirement to 2 + (11 X 6,438) = 70,820 and this exceeds the pre-V6 compiler limit.
Note: We will see later how the TEMPLATE keyword can be used in cases such as this to avoid wasting storage on the DeptData
and ProductData structures. You will see an example on the next chart.
© Partner400, 2015
Entering the Fourth Dimension
You can create as many dimensions as you like but ...
• Until V6 a single element at the top level DS could not exceed 64K
✦
✦
✦
In the example below changing Departments to DIM(11) broke the limit
See the notes page for the size calculations
Increasing Divisions to (say) DIM(99) has no impact
-
Other than the 16Mb storage limit for data items
dcl-ds Divisions
DivCode
Departments
end-ds;
Qualified Dim(5);
Zoned(2);
LikeDS(DeptData_T) Dim(10);
dcl-ds DeptData_T
DeptCode
Products
end-ds;
Template Qualified;
Zoned(3);
LikeDS(ProductData_T) Dim(99);
dcl-ds ProductData_T
ProdCode
MonthsSales
end-ds;
Template;
Zoned(5);
Packed(9:2) Dim(12);
Free Form
Version
More on the
TEMPLATE
keyword in a minute
Divisions(1).Departments(1).Products(1).MonthsSales = 0;
Notes - DS Size issues prior to V6
To see why the simple change of increasing the dimension of the Department DS array to 11 causes the storage requirement to
exceed the V5R4 size limit study the size of the nested DSs.
dcl-ds ProductData_T
Template;
ProdCode
Zoned(5);
MonthsSales
Packed(9:2) Dim(12);
end-ds;
ProductData is made up of the 5 byte Department code and a 12 element array. Each element is 5 Bytes long so the size of
ProductData is: 5 + (12 x 5) = 65 bytes.
dcl-ds DeptData_T
DeptCode
Products
Template Qualified;
Zoned(3);
LikeDS(ProductData_T) Dim(99);
end-ds;
DeptData consists of the 3 byte Department Code and 99 instances of the Products DS Array. So the size is: 3 + (99 X 65) = 6,438
bytes.
dcl-ds Divisions
DivCode
Departments
Qualified Dim(5);
Zoned(2);
LikeDS(DeptData_T) Dim(10);
end-ds;
Each element in Divisions consists of the 2 byte Division Code and 10 instances of the Departments DS array. So its size is: 2 + (10
X 6,438) = 64,382. This is under the V5R4 limit of 65,535, but not by much. Simply increasing the DIM on the Department DS array
to 11 raises the requirement to 2 + (11 X 6,438) = 70,820 and this exceeds the pre-V6 compiler limit.
© Partner400, 2015
Creating "Data Types" - V6's TEMPLATE
This code creates a new "Address" data type
• But no storage is allocated to it - it is just a "note" to the compiler
MailAddr & ShipAddr are defined using this data type
• They will contain the same field definitions as Address
• Because the LIKEDS keyword is used they are implicitly QUALIFIED
Place standard definitions like this in a /COPY member
• Then you can include ll of your standard definitions with one simple line
2+
3+
4+
5+
6+
7+
8+
9+
/COPY StdTypes
dcl-ds Address_T
Template;
Street1
char(30);
Street2
char(30);
City
char(30);
State
char(2);
Zip
zoned(5);
ZipPlus
zoned(4);
end-ds;
dcl-ds MailAddr
dcl-ds ShipAddr
LikeDs(Address_T);
LikeDs(Address_T);
Notes
Most modern programming languages include the ability to effectively create your own data types. V5R1 RPG gave
us the QUALIFIED and LIKEDS keywords. That was a first step along the path as they enabled us to define one DS
as looking exactly like another. V5R2 gave us the ability to use LIKEDS within a DS - allowing us to nest data
structures one within another.
In both cases however, the definitions to use memory and the only option which avoided this (using the BASED
keyword on the definition) precluded the use of initial values and could lead to errors if the programmer mistakenly
referenced one of the fields in the DS. If you are unfamiliar with this technique you'll find an example on a later notes
page.
With the new TEMPLATE keyword, RPG is all "growed up" and can now define templates for use by LIKE and
LIKEDS that do not use memory and can be given initial values. In fact you can even define files with the
TEMPLATE keyword - but more on that later.
© Partner400, 2015
"Data Types" - More ...
Illegal usage of templates is detected by compiler
• e.g. If you try to store anything in them
Initial values can be specified and can be cloned via Inz(*LIKEDS)
TEMPLATE can also be used with Stand-alone fields and files
• Templates can be referenced by %SIZE, %LEN, %ELEM and %DECPOS
2+
3+
4+
5+
6+
7+
8+
9+
/COPY StdTypes
dcl-ds Address_T
Template;
Street1
char(30);
Street2
char(30);
City
char(30);
State
char(2);
Zip
zoned(5);
ZipPlus
zoned(4);
end-ds;
dcl-ds MailAddr
dcl-ds ShipAddr
Free Form
Version
LikeDs(Address_T);
LikeDs(Address_T);
If MailAddr.City = 'Rochester';
MailAddr.State = 'MN';
EndIf;
Notes
You may see pre-V6 examples where templates are defined using the Based keyword. The only reason this was
done was to avoid wasting static memory on structures that would never be used to hold data - only to provide a
definition to be "cloned".
With V6's TEMPLATE keyword we now have an engineered solution to this problem so use it! DOn't use the old
method.
© Partner400, 2015
Sorting Subfiles
Want to sort subfiles?
Here's an example using many new features
The LIKEREC keyword
DS Arrays
I/O Operations using the result field
Naming Indicators
Notes
Requests relating to how to sort subfiles are one of those most commonly posed on Internet lists.
This simple demonstration program highlights a very simple approach that can easily be adapted to almost any
sorting requirement. The "secret" lies in loading the entire subfile into memory at the start of the process. Sorting it
and then redisplaying becomes not only a trivial task but also one that performs very well.
You may need to get to release 6.1 before you can define a subfile with 9,999 records in it - but you can adapt the
process to use a user space and simulate the array aspects. If you use a page-at-a-time approach to loading your
subfile, you will also have to adapt the logic so that the subfile is fully loaded before you attempt to display it after a
sort request - but this is not hard to do.
As you will see we are using many of the techniques we have described in this session in the program. Note that a
full source listing (including the file definitions) is included at the end of this handout.
© Partner400, 2015
Subfile Sort - Basic Flow
1. Load all records into the subfile array
2. Load the subfile from the array
3. Display the subfile
4. If sort requested determine which sequencing routine to use
5. Call the sort (qsort in this case)
6. Clear the subfile
7. Loop back to 2 until the user tells us we are finished
(1)
Count = LoadArray();
(2)
(3)
(4)
(5)
(6)
(7)
DoU (ExitRequest)
LoadSubfile(Count);
DisplaySubfile();
Select ...
Sort ...
ClearSubfile();
EndDo;
// Store count of records loaded
// Copy data from array into subfile
// Determine sort sequence
// Clear out subfile ready for reload
Notes
A common user request is to sort subfile data on the screen. Using this sorting technique, this can be
fairly simply accomplished. The data is first loaded into the array, then the subfile is loaded from the
array. If/when the user requests a different sequence to the data in the subfile, don't retrieve it again
from the file. Instead simply re-sequence the array data using the simple SORTA technique and reload
the subfile from the array.
The biggest advantage to this approach is that additional sort options can be added to a program in a
matter of minutes. Add the indicator to the display file, write the sort routine, add the extra two lines to
the SELECT clause and you are pretty much done.
This method is of course only good for data that does not need to be up to the minute. Since the data
is in the array any changes being made to the data in the database will not be reflected. In our
experience though, the user finds it much less confusing if the data is the same after a sort as they saw
previously - it confuses them if they sort something into a different sequence and then can't find the
record they were looking at before!
If it is vital for the data to always be current, then using embedded SQL with an appropriate ORDER
BY clause is a better way to go.
© Partner400, 2015
1. The LoadArray Routine
The Read loads data directly into the ProductRec DS
• ProductRec itself is defined using LikeRec(PRODUCTR)
Eval-Corr then populates the subfile array element
• Which itself was defined with LikeRec(ProdSfl: *Output)
This example uses a single file and no calculations
• But as you can imagine any logic you like can be added to the load routine
Dcl-Proc LoadArray;
Dcl-PI
*N Int(10);
Dcl-DS ProductRec LikeRec(PRODUCTR)
Dcl-S n
Int(10);
// Read all records and load array of subfile records
DoU %EOF(Product);
Read ProductR ProductRec;
If Not %Eof(Product);
n +=1;
Eval-Corr SubfileRec(n) = ProductRec; // Load subfile rec
EndIf;
EndDo;
Return n; // Return record count
2. The LoadSubfile Routine
If this were any simpler it would have written itself!
RecordCount is passed in as a parameter
And used to control the For loop
Write uses the subfile DS array element as the data source
With the RRN being used as the array subscript
Dcl-Proc LoadSubfile;
Dcl-PI
*N;
Dcl-S RecordCount
Int(10);
For RRN = 1 to RecordCount;
Write ProdSfl SubfileRec(RRN);
EndFor;
End-Proc;
© Partner400, 2015
4 & 5 Sort Selection Portion of Mainline
Note use of Named Indicators for testing input F key requests
SortDS is the name of the prototype for qsort()
(4)
Select;
// Determine sort option selected by user
When ( SortPrCode );
SortDS(SubfileRec(1):
// Pass address of first element
Count:
%Size(SubfileRec):
%PAddr( SortProduct ); // Procedure for product code sort
When *In(SortDesc);
Count:
%Size(SubfileRec):
%PAddr( SortDescr );
Other;
// Default to Product sort
Count:
%Size(SubfileRec):
%PAddr( SortProduct );
EndSl;
(5)
(5)
(5)
5. The SortProduct Routine
The only difference between the sorts is the field comparison
Unlike SORTA - qsort() can handle multiple key sorts
You just write the RPG code to do it!
We use LikeRec to define the parms
The qualified individual field name ( e.g. Element2.ProdCd ) can then be
used in the comparisons
Dcl-Proc SortProduct;
Dcl-PI
*N Int(10);
Element1 LikeRec(ProdSfl: *Output)
Element2 LikeRec(ProdSfl: *Output)
Select;
When Element1.ProdCd > Element2.ProdCd;
Return High;
When Element1.ProdCd < Element2.ProdCd;
Return Low;
Other;
Return Equal;
EndSl;
End-Proc;
© Partner400, 2015
A Sample Multi-Key Sort Routine
Unlike SORTA qsort() can use any logic you like for sequencing
The most commonly used variant is to accommodate multiple keys
• e.g. To sort City in State as in this example
Dcl-Proc CityInState;
Dcl-PI
*N Int(10);
Element1 LikeRec(CustAddress: *Output)
Element2 LikeRec(CustAddress: *Output)
Select;
When Element1.State > Element2.State;
Return High;
When Element1.State < Element2.State;
Return Low;
When Element1.City > Element2.City;
Return High;
When Element1.City < Element2.City;
Return Low;
Other;
Return Equal;
EndSl;
End-Proc;
Any Questions ?
?
Please e-mail Jon at:
for any questions
© Partner400, 2015
Meeting Users' Needs
with Free Software
for your IBM i
Jon Paris
www.Partner400.com
Notes
Jon Paris is co-founders of Partner400, a firm specializing in customized education and mentoring services for IBM i
(AS/400, System i, iSeries, etc.) developers.
Jon's career in IT spans 40+ years including a 12 year period with IBM's Toronto Laboratory.
Together with his partner Susan Gantner, Jon devotes his time to educating developers on techniques and
technologies to extend and modernize their applications and development environments. Together Jon and Susan
author regular technical articles for the IBM publication, IBM Systems Magazine, IBM i edition, and the companion
electronic newsletter, IBM i EXTRA. You may view articles in current and past issues and/or subscribe to the free
newsletter at: www.IBMSystemsMag.com.
Jon and Susan also write a weekly blog on Things "i" - and indeed anything else that takes their fancy. You can find
the blog here: ibmsystemsmag.blogs.com/idevelop/
Feel free to contact the author at:
© Partner400, 2015
Free Software ?
By "Free" I mean as in:
• No need to obtain budget approval
• No direct cost
✦
Although you may choose to opt for a service contract
• Free to modify to meet your own requirements
✦
In most cases but not all
Note: "Free" does not mean "Cheap and Nasty"
• Much of the free software I have used is as good or better than
software from conventional sources
• For many ISVs this is just a different marketing strategy
Notes
The first thing you need to understand is that free does not have to mean low quality. I have seen many free software
packages that exceed in quality and capability those that are sold for substantial amounts of money.
Many of the free packages are available with maintenance contracts so that you have someone to call if you have
problems. This also helps if the idea of no (obvious) line of support makes your management nervous.
Many of the packages are actually a subset of a fuller featured commercial offering and used as a form of marketing.
In other words giving the base software away instead of spending $s on advertising, marketing reps, etc.
© Partner400, 2015
Meeting Users Needs ?
I'm using a very broad definition here of the term "Meeting"
• What I'm really talking about is ...
Anything that will help you to say "YES" to user requests
• Part of our problem is that we say "No" far too often
• Our counterparts on Windows etc. don't say that as often ...
✦
✦
So guess who gets to do all the new interesting stuff
And who is left defending their platform and maintaining old stuff
-
As well as dealing with interfacing their new stuff with our old stuff!
So we'll talk about everything:
• From tools that can help document systems
• To tools that help you build apps faster
• To complete applications that fill holes in your portfolio
Notes
For a long time now I have been trying to persuade IBM i shops to "just say Yes". It is all too common to see IBM i
being displaced because the staff simply cannot react fast enough to the needs of the business.
Switching to Java is not the answer, and switching to a WIndows "solution" isn't either. The answer, in my opinion, is
to make more use of free and low cost tooling to give the users what they want. Not everything can be done for free
of course, but it is far easier to get budget approval for staff and/or expensive tooling when your users are happy with
what you do and supportive of your efforts.
Tools such as the ones we will discuss here can help you achieve this. What's even better is that many can even be
installed and experimented with on your PC before you deploy them to your IBM i.
© Partner400, 2015
What We’ll Cover ...
Updating business practices with new free/low-cost tools
• Everything from:
✦
✦
✦
✦
Customer Relationship Management
Help Desk software
User Documentation
Web sites
Extending your RPG application set
• Free Tools that can help you to:
✦
✦
Build new applications faster or
Extend existing applications into new areas
A Few Examples From the PHP World
Customer Relationship Management
Portal
Content Management
e-Commerce
Content Management
Course Management System
Bulletin Board
© Partner400, 2015
Simple Wiki
Wiki
Help Desk
SugarCRM - Customer Relationship Management
www.sugarcrm.com
MANTIS
Help Desk Software
• http://mantis400.com
Uses native DB2
• Data usable from RPG, Query, COBOL, SQL...
Available under GNU GPL Open Source license
• I.e. It is FREE FOREVER
Create/manage Internal and External Support Issues
• Any web browser, any client operating system
• E-mail notification as Issues are progressed
Supported by Curbstone Corporation
• They have used it in-house to track problems for many years
© Partner400, 2015
Mantis Main View
Not as sophisticated
as SugarCRM
• But highly
customizable
• And the price is
tough to beat!
osTicket - Support Ticket Tracking
An alternative to Mantis - osticket.com
• Definitely better looking
• Doesn't track as deeply into the development process
✦
More oriented to customer facing problem reporting
• Requires use of Zend DBi (MySQL) or you can convert to DB2
© Partner400, 2015
Yet Another Help Desk Option
Hesk - The system currently used by System i Developer
www.hesk.com
• Simple
• Free
• Well supported
What more could
you ask!
Part of a complete
CRM suite
Communication - Internal and External
Internal and External Web sites
• A web site that is easy to maintain is more likely to be maintained
• Consider using a Wiki, or Content Management System (CMS)
I'll be discussing MediaWiki
• It is so easy to use that your end users can write their own pages!
For a CMS how about Drupal?
• The latest UI changes significantly simplify implementation
How about a Bulletin Board system?
• PHPBB is used by huge numbers of Bulletin Boards all over the
world
✦
So customizable that you may not recognize them all as being the same
software
• A great way to share hints, tips, or news of that great new
subprocedure
• Can also be used as a low-level Help-Desk vehicle
✦
Or as part of a program to gather feedback and communicate with customers
© Partner400, 2015
Wikis - MediaWiki (mediawiki.org)
Sample PMWiki Coding
...
You can also do '''bold''' text and ''italic'' but '''''bold italic''''' is really messy! But
tables are quite easy to do as you can see in the example below.
{|
|+ Really Simple Table
| Item A1 || Item A2 || Item A3
|| Item B1 || Item B2 || Item B3
...
Indented lists are also very simple and can include numbered lists.
*First Level [[LinkPage | with a built in link]]
**This at the second level
**So is this
***Indent a bit more
***And another at that level
*Now back to the top level
And here we are back in normal text followed by an example of a slightly more complex table with
row and cell alignment:
{| class="wikitable"
|+ More Complex Example WIth Headers and Alignment
! < ---- Header 1 ---- >
!! < ---- Header 2 ---- > !! < ---- Header 3 ---- >
|| Default cell 1 || Default cell 2 || Default cell 3
|- style="text-align:right;"
| Row right align 1 || Row right align 2 || Row right align 3
||Left aligned || style="text-align:center;"| Centered || style="text-align:right;"| Right aligned
|}
© Partner400, 2015
Build Web Sites - Drupal & Drupal Gardens
Drupal (drupal.org)
• Complete Content Management System
✦
✦
Features include support for blogs, forums, RSS feeds, etc.
Simplified install process, Language support, Update status and more
• Version 7 had a heavy focus on usability
✦
✦
Dealing with many earlier complaints about ease-of-use
Comprehensive support for developers brings it really close to being usable
as an Application Framework
• Version 8 promises to be even better
Drupal Gardens gives you a way to try it all out easily
• You can build and run a complete site free-of-charge
✦
Samples www.test-drupalgardens.edrupalgardens.com/taxonomy/term/41/0
• Then export it to run it locally on your own system
✦
Or upgrade at very low cost to run it on their web site
Drupal - Simple Site
Editing options
only appear for
authorized users
© Partner400, 2015
And Now ...
Extending your RPG application set
• Free Tools that can help you to:
✦
✦
✦
Test applications more easily
Build new applications faster
Extend existing applications into new areas
Notes
In this section we will look at tooling that can help you build new applications faster. We find that remaining relevant to your organization is the best way to guarantee your future. Simply being
"indispensable" due to knowledge of a specific piece of software is no good if that package is kicked out an replaced!
© Partner400, 2015
Generating Test Data With Faker
<?php
require __DIR__ .'/../src/autoload.php';
$faker = Faker\Factory::create();
print '<table border="1">';
for ($i=0; $i < 10; $i++) {
print '<tr><td>' . $faker->firstName . ' ' . $faker->lastName . '</td>';
print '<td>' . $faker->phoneNumber . '</td>';
print '<td>' . $faker->streetAddress . '</td>';
print '<td>' . $faker->city . '</td>';
print '<td>' . $faker->postcode . '</td>';
print '<td>' . $faker->stateAbbr . '</td></tr>';
}
print '</table>';
Find it at github.com/fzaninotto/Faker
• Names, Addresses, Phone numbers, Zip codes, eMails, Text, etc.
Also try generatedata generatedata.com
• An on-line offering that produces downloadable test data
Notes
The sample code is producing an HTML table for demonstration purposes. But it could equally produce a CSV file, or
for that matter update a test database directly. The table below is an example of the output of this script.
© Partner400, 2015
Building New Applications Faster
Courtesy of Application Frameworks
• Acknowledgement that applications have common requirements
• A Framework is a set of sophisticated building blocks
✦
Think of them as providing a more advanced version of program cloning
• Most employ Object Oriented design and programming methods
✦
But you don't have to code in an OO style to use them
A great way to produce feature-rich applications quickly
• Three that I have used and like are:
✦
✦
✦
PHPGrid
XataFace
and iDataGrid - an IBM i specific offering
• We will look briefly at each of them in a moment
PHPGrid - www.phpgrid.com
New tool that simplifies the building of CRUD applications
• Can be used by itself
Not quite
free!
• Or embedded in more complex applications
Completely OO but still easy to use without OO skills
• Only lines marked
are required for a working grid
require_once("../conf.php"); // Include main phpgrid configuration and routines
// Identify table and required columns and default sort column
$dg = new C_DataGrid("select customerNumber, customerName, addressLine1,
addressLine2, city, state, country
from customers", "customerNumber", "customers");
$dg -> set_query_filter("state != '' ");
// Set WHERE clause
$dg -> set_dimension(1000, 800);
$dg -> set_pagesize(40);
// Set height and width of grid
// Increase page size to 40
$dg -> enable_search(true);
$dg -> enable_export('PDF');
// Enable search
// Enable pdf export
$dg -> display();
// And now show the world!
© Partner400, 2015
Results of Simple Script
Search and
PDF Generation
XataFace - www.Xataface.com
Pronounced Zataface
• Originally developed for use at Simon Fraser Univerisity
✦
Faculty of Applied Science
Designed for the development of Data Centric Applications
• Separates the logic from the presentation
• Inherits relationships, sorting, searching etc. from the database
✦
Any fields or relationships added to the database are immediately reflected in the application
- often with no additional work
Basic Process is:
• Design the database
• Optionally decorate the application
✦
Done by setting parameters
• Optionally design the web pages
✦
✦
i.e. Modify the templates used
The equivalent of the display file DDS
© Partner400, 2015
Simple Xataface Application
This is a simple four table application
• Customer Orders, Customers, Sales Reps, and Products
It uses Xataface's default look-and-feel
• Sorting, searching, paging, import, export, etc. capabilities are all built in
Structure of a XataFace Application
The top level folder JonstestApp contains the application
• conf.ini contains connection information for the database and
identifies the tables to be used
• index.php is the main driver script - as you will see it is very simple
• Directory tables contains one directory for each table to be
"decorated"
✦
More on these files in a moment
© Partner400, 2015
Source Code conf.ini and index.php
; File: conf.ini
; Description:
;
Configuration details including database connection
;
and tables to be included in the application.
[_database]
host = "localhost"
user = "dbuser"
password = "dbuserpw"
name = "sampledb"
[_tables]
orders = "Customer Orders"
orderDetails = "Order Details"
customers = "Customers"
employees = "Sales Reps"
/* File: index.php
/* Description:
/*
First line incorporates the XataFace Library
/*
Second invokes the display method that sets everything in motion
<?php
require_once '/Applications/MAMP/htdocs/xataface/dataface-public-api.php';
df_init(__FILE__, 'http://localhost:8888/xataface')->display();
?>
Source Code - fields.ini and valuelists.ini
Directory tables contains one directory for each table to be "decorated"
•
✦
✦
My directory customer contains these two files
fields.ini provides information about the individual fields
valuelists.ini contains the data needed to build pull-down lists etc.
This is the only customization I have done - everything else is standard
[customerNumber]
widget:label = "Customer Number"
widget:description = "Enter the Customer Number:"
[customerName]
widget:label = "Customer Name"
widget:description = "Enter the Customer's Name:"
[salesRepEmployeeNumber]
widget:label = "Sales Representative"
widget:description = "Enter the Sales Rep's Code:"
widget:type=select
vocabulary=salesrep
[salesrep]
__sql__ = "select employeeNumber, lastName from employees order by lastName"
© Partner400, 2015
iDataGrid - A New Boy in Town
iDataGrid - The Application Wizard
© Partner400, 2015
WOW - Community Edition
Java rather than PHP based
• Tool is free but no source code supplied
• If you like it you can grow into the full commercial version
-
www.planetjavainc.com
While on the Subject of Java Based Tools
There's a new kid in town!
• Open Legacy (openlegacy.com)
This is a re-facing/ re-purposing tool set
• Comes complete with a graphical designer for 5250 modernization
• Also has tooling to assist in building web services etc.
I have not yet had time to "play"
• If I like it, expect to see an article in the future
✦
And maybe even a session at the RPG & DB2 Summit
© Partner400, 2015
Shopping For Free & Low-Cost Software
Where to "Shop"?
• Other than any specific sites mentioned ...
Hotscripts.com
• Great place to hunt for utilities, applications, and more
• Pay attention to the Pepper Ratings and whether the package is
regularly updated
For PHP tools and articles look to PHP Publications
✦
Sitepoint's PHP Master zone (sitepoint.com/php/)
-
Sitepoint also have great books and classes on all things web
• Web and PHP Magazine
✦
Free download (webandphp.com)
Questions? Comments?
Please feel free to contact me:
?
© Partner400, 2015

Notes - Gateway/400 Group

Transcription

Similar documents

Leisure activities and relaxation

Batch Production of Driving Distances and Times Using SAS® and

February - Stapleton

MIDI-PYRENÉES

United States Air Force Academy Summer Seminar

Giving yourself all the chances to succeed... in college !

August 2015 Newsletter - Opticians Alliance of New York

Case Study: GateWay Community College

AG1977-A4-1-1-3-002-jpeg - Historical Papers

Untitled