{ From user Drice, Model Parsing_XML_data at 14-Mar-2016 4:10:59 PM, encoding="UTF-8" }
SoftwareVersion 4.6.1
{ System Variables with non-default values: }
TypeChecking := 1
Checking := 1
SaveOptions := 2
SaveValues := 0
NodeInfo FormNode: 1,0,0,1,0,0,0,,0,0,,0
Model Parsing_XML_data
Title: Parsing XML data
Description: This example demonstrates two methods for parsing data in an XML form, to get data from XML into your model. In the example, I extract a 2-D array of data -- which is just part of what is in the XML~
~
The first method uses a full XML parser, one provided by Microsoft. Using this requires the Enterprise edition of Analytica, since is makes use of COM integration.~
~
The second method uses regular expressions. For simple data files, this can be easier, but the regular expressions to do necessarily have a full awareness of the full XML syntax.~
~
The same general steps were followed in both cases, for easy comparison. In both cases, row and col indexes are defined to match the actual number or rows and cells per row in the data.
Author: Dale Rice~
Lumina Decision Systems
Date: Mon, Mar 14, 2016 9:58 AM
SaveAuthor: Drice
SaveDate: Mon, Mar 14, 2016 4:10 PM
DiagState: 2,1,0,960,585,17
WindState: 2,772,93,720,350
FontStyle: Arial,15
FileInfo: 0,Model Parsing_XML_data,2,2,0,0,W:\Tech Support\Parsing XML example.ana
Variable XML_Text
Title: XML Text
Description: The XML text that we will extract data from.
Definition: '~
~
~
~
~
| ~
~
| ~
~
~
| ~
~
~
| ~
~
~
| ~
~
~
| ~
~
~
~
| ~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
| ~
~
~
| ~
~
~
~
| ~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
| ~
~
~
~
| ~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
| ~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
| ~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
| ~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
| ~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
| ~
~
~
~
| ~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
| ~
~
~
~
| ~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
~
| ~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
| ~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
| ~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
~
~
~
~
~
| ~
~
~
| ~
~
~
'
NodeLocation: 120,96,1
NodeSize: 48,24
ValueState: 2,728,84,844,639,,MIDM
Aliases: Alias Al1457994131, Alias Al1457994171
Module Using_MSXML_to_parse
Title: Using MSXML to parse
Description: Demonstrates the use of the MSXML parser to access the parsed XML. The MSXML DOM component has the complete XML tree, and enables a rich set of access functions to extract data from the XML. ~
~
To use the DOM parser, you do need to have the Analytica Enterprise edition or better, since is uses the COM Integration features.
Author: Dale Rice
Date: Mon, Mar 14, 2016 4:10 PM
NodeLocation: 272,96,1
NodeSize: 48,32
DiagState: 2,642,284,960,585,19
WindState: 2,814,14,720,350
Variable xmlDoc
Title: xmlDoc
Description: Instantiate an XML DOM parser, and load the XML into its memory.~
In addition to requiring Analytica Enterprise edition or better, you need to have the Microsoft MSXML Core Services installed on your computer (which is pretty common).
Definition: var d := COMCreateObject("Msxml2.DOMDocument.3.0");~
d->async := false;~
d->LoadXML(XML_Text);~
if (d->parseError->errorCode <> 0) Then Error(d->parseError->reason);~
d
NodeLocation: 200,48,1
NodeSize: 48,24
ValueState: 2,120,126,416,303,,MIDM
Index Row
Title: Row
Description: Indexes the rows of data read from the XML.~
~
This is computed by parsing the XML and finding how many row tags there are.
Definition: ComputedBy(xmlRow)
NodeLocation: 336,48,1
NodeSize: 48,24
ValueState: 2,1100,30,416,303,,MIDM
Variable xmlRow
Title: xmlRow
Description: Extract the row nodes from the XML.~
Set the Row index to the appropriate size.
Definition: Var nodes := xmlDoc->selectNodes("//*/row");~
Row := 1..nodes->length;~
nodes->item(Row-1);
NodeLocation: 464,48,1
NodeSize: 48,24
Variable n_Cells
Title: n Cells
Description: Count how many cells are in each row. This is done to determine how long to make the Col index.
Definition: xmlRow->selectNodes("cell")->length
NodeLocation: 584,48,1
NodeSize: 48,24
Index Col
Title: Col
Description: It is possible that each in the XML has a differing number of cells. If so, you need to decide how many columns to include in your final data set. I decided to use the shortest row, so that all columns would be as dense as possible.
Definition: 1..Min(n_Cells,Row)
NodeLocation: 336,128,1
NodeSize: 48,24
WindState: 2,112,558,720,350
Variable xmlCell
Title: xmlCell
Description: Extract the nodes in each column. Because we've already created the Col index, and we use this in the parameter to Item(...), this indexes the result by our Col index, so that we have the dimensionality we want here. Each COM object in the result is an IXMLDOMNode object, corresponding to a tag.
Definition: xmlRow->selectNodes("cell")->item(Col-1)
NodeLocation: 464,128,1
NodeSize: 48,24
ValueState: 2,312,318,621,304,,MIDM
Variable fill_color
Title: fill color
Description: Extract the fill color from the cell node.
Definition: xmlCell->selectSingleNode("fill/@color")->text
NodeLocation: 584,128,1
NodeSize: 48,24
ValueState: 2,839,118,796,324,,MIDM
NodeColor: 65531,39321,65535
Alias Al1457994131
Title: XML Text
Definition: 1
NodeLocation: 80,48,1
NodeSize: 48,24
Original: XML_Text
Close Using_MSXML_to_parse
Module Parse_with_regular_e
Title: Parse with regular expressions
Description: Demonstrates the extraction of data from XML using regular expressions.~
~
The regular expressions do not perform a complete or valid parse of the full XML, but instead simple locate the data of interest by finding the relevant patterns. For data files with a simple structure (as the majority of data files often have), this can the easiest and most immediate approach, especially if you are already familiar with regular expressions.~
~
This approach works in any edition of Analytica.
Author: Dale Rice
Date: Mon, Mar 14, 2016 4:10 PM
NodeLocation: 272,176,1
NodeSize: 56,32
DiagState: 2,526,310,960,585,19
WindState: 2,689,384,720,350
Alias Al1457994171
Title: XML Text
Definition: 1
NodeLocation: 120,72,1
NodeSize: 48,24
Original: XML_Text
Variable Row_XML
Title: Row XML
Description: The first step is to break the file into the individual rows -- the XML bracketed by ... .~
Simultaneously, this sets a global index to make the indexing of rows more direct, which still being determined by the actual number of rows that appear in the XML.
Definition: var splitRows := FindInText("(.*?) | |
",XML_Text,return:'S',re:1, subpattern:1,repeat:True);~
Cell_row := CopyIndex(splitRows.repeat);~
splitRows[.Repeat=Cell_row]
NodeLocation: 264,72,1
NodeSize: 48,24
WindState: 2,102,86,935,350
ValueState: 2,440,446,457,511,,MIDM
Index Cell_row
Title: Cell row
Description: Index for each row in the XML. Set after matching to how many tags appear in the XML.
Definition: ComputedBy(Row_XML)
NodeLocation: 120,152,1
NodeSize: 48,24
Variable Num_cells_by_row
Title: Num cells by row
Description: Before creating a column index, we need to determine how long to make it. To do this, we need to count how many | tags appear within each row tag. We collect these -- hopefully there are the same number in each row, but that isn't necessarily guaranteed.
Definition: Var rxml[] := Row_XML Do size(FindInText(" tag, indexed by our row and col indexes.
Definition: FindInText('(.*?) | ', Row_XML,re:true,return:'S',repeatIndex:cell_col,subpattern:1)
NodeLocation: 400,72,1
NodeSize: 48,24
WindState: 2,96,548,720,350
ValueState: 2,1057,332,455,434,,MIDM
Variable cell_fill_color
Title: cell fill color
Description: The final data of interest extracted from the XML, and indexed as a 2-D array as desired, with the global indexes defined also in the process.
Definition: FindInText('