guide

 TableofContents
1 Introduction ............................................................................ 3 2 How to create a new qLoader project .................................... 4 3 Loading your data files ........................................................... 5 3.1 How to load text files located on a file system folder ................................... 5 3.1.1 Before your start ................................................................................................................ 5 3.2 Loading data from FTP or HTTP remote files ................................................. 9 3.3 How to load remote data from an S3 bucket (Amazon Web Service –AWS)11 3.4 Loading data from Google SpreadSheet ...................................................... 13 4 Load from databases ............................................................ 16 4.1 How to load from a PostgreSQL database ................................................... 16 4.1.1 Using native database connection ................................................................................... 16 4.1.2 Using ODBC database connection .................................................................................... 17 4.2 How to load from a MySQL database ......................................................... 18 4.2.1 Using native database connection ................................................................................... 18 4.2.2 Using ODBC database connection .................................................................................... 18 4.3 How to load from an Oracle database ........................................................ 20 4.3.1 Using native database connection ................................................................................... 20 4.3.2 Using ODBC database connection .................................................................................... 20 4.4 How to load from a MS SQL Server ............................................................. 22 4.4.1 Using native database connection ................................................................................... 22 4.4.2 Using ODBC database connection .................................................................................... 22 4.5 Table creation with a database connection ................................................ 23 5 Synchronize your new tables with BIRT Analytics ............... 26 6 Providing data access privileges for users ........................... 27 1 Introduction This document is a step‐by‐step guide that shows how to connect BIRT Analytics to distinct sources of data (flat text files, databases, etc.). You will learn how to load your own data into our FastDB engine and start working immediately, taking advantage of the power and ease‐of‐
use of BIRT Analytics. The guide focuses on the configuration of qLoader (BIRT Analytics data loading tool) to allow you to load your own data. BIRT Analytics permits the loading of text files located on a/an: 



File system folder FTP / HTTP server Amazon S3 Google Spreadsheet BIRT Analytics also lets you load directly using a native connection or using an ODBC connection from certain databases, such as: 



PostgreSQL MySQL Oracle Microsoft SQL Server There are five basic operations that must be performed to successfully load your data. They are found in: Chapter 2 ‐ Before any files can be loaded, you must first create your new project in the BA loading tool as described in this chapter Chapter 3 ‐ Once you have created your new project, you can first follow the steps for loading your files into your new project and then proceed to the steps for loading your project into the BA fast database. Chapter 4 ‐ This chapter provides the steps for loading files from another database. Chapter 5 – To be able to use your data, you must synchronize the new repository (containing your data) with BIRT Analytics – as shown in this chapter. Chapter 6 ‐ This chapter shows you how to grant user permissions for new project data. This needs to be done even if you are the only user. Note: It is possible to connect BIRT Analytics and BIRT Reports / Dashboard. 3 of 28 2 How to create a new qLoader project 1. When you log in to BIRT Analytics Loader for the first time, there are no existing projects. Click “New” to open the “Project management” window. 2. Set the following parameters in the “Create new Project” window and click “OK”. (see example screen below): o
o
o
o
o
o
Name: type the name of the project. IP: type the IP address of the web server where BIRT Analytics Loader is installed. Port: type the port number that the BIRT Analytics Loader application uses. The default port number is 8105. SSL: Choose SSL if a secure connection is used and set the port number as 8106. User: type the FastDB Engine user. The default user is “SYSTEM”. Password, type the FastDB Engine user’s password. The default password is “PASSWORD”. 4 of 28 3. The “Project management” window appears again (see above – step 1). Click on the arrow to find your new project name and to display it in the project field. Next click “Open” if you want to start working on defining your new project. 3 Loading your data files 3.1
How to load text files located on a file system folder 3.1.1 Before your start Defining your new project file for text files will require entering file information such as Character Encoding and the End Line Method. You can get this information by opening your files with a text editor such as Notepad++ In the screenshot below, the left red box indicates the End Line Method (Dos\Windows) and the right red box shows the Character Encoding (UTF‐8). 5 of 28 The following steps show you how to load your new project with text files located on a file repository : 1. Create a new qLoader Project called “LoadTextFile” (as shown in the procedure above). Then open it and click on the “Data” Tab. 2. You are now ready to upload the desired text files into your qLoader project. In the “Data” tab click on the “Upload file” icon found at the bottom of the tab (see below). 3. This opens a window inviting you to select the desired file in your directory. Once you have selected the file, click the “Open” button to start the upload process. After that, you can see your chosen file in the “Files” panel along with its size. (see example below). 6 of 28 4. Now that your data file is in your project, you need to define the BIRT Analytics database table on which you want to load the data. For this, click on the “New from file” button. The table declaration window appears: 5. You must complete the following fields: o Database: name of the database where you want your file to be allocated o Name: table name o Action: for complete loads, check “Create”, for incremental loads check “Append”. o Overwrite: check this option if you are planning to execute the project more than once. o Type: depends on the file type selected o Delimited: each field is delimited with a special character (comma, pipe, quote, etc.). o Flat: if each field has a fixed length o Encoding: select the proper character encoding, you can use a text editor, as explained above, to know the file character encoding of your file. o End Line: select the proper end line used on the file. Normally CrLf for Windows Systems, Cr for Unix Systems. o File: select the file you want to load into BIRT Analytics database. If you expand the list, you can see all the files previously uploaded into qLoader. o Header: select if the file has a header row. o Delimiter: in case of a delimited file, select the proper delimiter character. For a delimited file (recommended), you can now obtain the names and types of all of its columns by simply clicking on the “Get Columns” button. This shows you all the columns identified by qLoader. 7 of 28 6. You can change any column type, width or format, if the one assigned by qLoader is not correct. See example below: 7. Click “OK” to see your newly defined table ready to be loaded into BIRT Analytics. If you have more files to load, repeat the process for each one of them. 8. Finally, click “Save” in the main menu to save the project. Before you can run your new project, you first need to synchronize the new repository containing your data with BIRT Analytics. Go to Section 5 of this document to learn how to do this 8 of 28 3.2
How to load data from FTP or HTTP remote files Note: Before starting, you need to obtain the complete route of your file in the FTP or HTTP server. For this example we will use: http://localhost/Test_RDP_FTP_HTTP.csv . You will also need file information such as, among others, Character Encoding and the End Line Method (see Section 3.1.1 for help). The following procedure shows you how to load a remote text into BIRT Analytics: 1. Create a new qLoader project as shown in section 2.1 of this document. (Our sample project is called “CustomerRemoteFile”). Open the new project. Next open the “Data” tab and click on the “Create a new Remote Data Provider” icon on the bottom of the tab: 2. In the “URL” window that opens, enter your project name in the “Name” field. Our sample project is called “CustomerRemoteFile”. Next paste your URL link (or the sample one obtained in step 1) into the URL field. Then click “OK” to close the “URL” window and go back to the “Data” tab. 3. Click on the “New from file” button in the “Data” tab options panel, to open the “Table declaration” screen. 9 of 28 4. Fill in the required fields in your new “Table declaration”. Note: Here you need to enter the file information for the Encoding, Endline, Delimiter and Qualifier fields as well as choosing “header” if applicable. Then define your file by selecting the remote data provider (created previously) from the dropdown file list. In our case we chose “CustomerRemoteFile”. 5. Now click the “GetColumns” button in the “Table declaration” screen to see your Remote file columns and make any changes in column type, width or format, if necessary. When finished click “OK” to go back into the main screen. (If you have more files to load, repeat the procedure for each one of them.) 10 of 28 6. Now back in the main screen, you can click “Save” to save your project. Before you can run your new project, you first need to synchronize the new repository containing your data with BIRT Analytics. Go to Section 5 of this document to learn how to do this. 3.3
How to load remote data from an S3 bucket (Amazon Web Service –AWS) Note: Before starting, you need to obtain the complete route of the file in the S3 bucket. For our example we have chosen ‐ http://my‐s3‐bucket‐name.s3.amazonaws.com/Test_RDP_S3.csv . You will also need other file information such as, among others, Character Encoding and the End Line Method (see Section 3.1.1 for help). Follow the steps to load files from an Amazon Web Service into BIRT Analytics: 1. Open qLoader and create and save your new project.” (as shown in Section 2.1). Our sample project is called “CustomerRemoteS3File”. 2. Open the new project. In the “Data” tab, click on the “Create a new Remote Data Provider” icon at the bottom of the tab. 3. In the URL screen that opens, insert a descriptive name for your new Remote Data Provider. Paste your link into the URL field. (In our sample we use “CustomerRemoteS3File” and: http://my‐s3‐bucket‐
name.s3.amazonaws.com/Test_RDP_S3.csv). Click “OK”. 11 of 28 4. Back in the “data” tab, click on the “New from file” button in the options panel (see below), to open the “Table declaration” screen. 5. Fill in the required fields in the “Table declaration” that opens. Note: Here you need to enter the file information for the Encoding, Endline, Delimiter and Qualifier fields as well as choosing “header” if applicable. Then define your file by selecting the remote data provider (created previously) from the dropdown file list. In our case we chose “CustomerRemoteS3File”. 6. Now click the “GetColumns” button in the Table declaration screen to see your Remote file columns and make any changes in column type, width or format, if necessary. Click “OK”. 12 of 28 Now back in the main screen, you can click on “Save” to save your project. Before you can run your new project, you first need to synchronize the new repository containing your data with BIRT Analytics. Go to Section 5 of this document to learn how to do this. 3.4
Loading data from Google SpreadSheet In order to load a Google SpreadSheet into BIRT Analytics you should follow the next steps: 1. Open the desired SpreadSheet on Google Docs and click on the “Share Button”: 2. From the Share Windows, click on the “Copy Link” button and then click “Done”. 3. Create a new project in qLoader (as shown in Section 2.1 of this document). In our example we called our project “CustomerSpreadSheet”. 13 of 28 4. Open the new project. In the mains screen “Data” tab, click on the “Create a new Remote Data Provider” icon at the bottom of the tab. 5. This opens the URL screen where you need to enter a descriptive name for your new Remote Data Provider. We use “CustomerSpreadsheet” in our example below. Next paste the link you obtained from the Google SpreadSheet (in Step 2) into the URL field. 6. Now you need to modify this freshly pasted url to obtain: https://docs.google.com/spreadsheets/d/eiunaosi398894nd93dnmeh/export?format=csv&usp=sharing To do this we replaced the “edit” word by “export” and included the text “format=csv&” after the char “?”. Click “OK” once the new URL is correctly defined. 7. Now back in the main screen. Click on the “New from file” button in the “Data” tab to declare your new table in the “Table declaration” screen (shown below). You need to fill in the fields. This includes selecting the following:  Encoding: ASCII  Header: Checked  EndLine: “CrLf” 14 of 28 

File: From the file list, select the remote data provider created previously. Delimiter: , 8. Then click the “GetColumns” button to see the defined columns and edit their type, width or format, if necessary. When finished, Click “OK”. Now back in the main screen, you can click on “Save” to save your project. Before you can run your new project, you first need to synchronize the new repository containing your data with BIRT Analytics. Go to Section 5 of this document to learn how to do this. 15 of 28 4 Load from databases BIRT Analytics lets you load directly from certain databases, using a native connection (installing the client software of the database) or using an ODBC connection (installing the client or ODBC driver). How to load from a PostgreSQL database 4.1
4.1.1 Using native database connection BIRT Analytics is able to connect to a PostgreSQL database using client software: Obtaining the client software: 

http://www.postgresql.org/download/windows/ http://www.enterprisedb.com/products‐services‐training/pgdownload#windows In this case we suggest using: postgresql‐9.0.18‐2‐windows‐x64.exe Installation procedure: 1. Execute the downloaded package 2. Add C:\Program files\PostgreSQL\9.0\bin into PATH 3. Restart server qLoader configuration procedure: Set the following parameters for the native database configuration: 




Type: postgresql Server: host address of the server where database server is located Port: where database service is listening in the server (usually 5432) User and Password: of the database service. Database: database where it is located the data we want to load in BIRT Analytics 16 of 28 4.1.2 Using ODBC database connection BIRT Analytics is able to connect to a PostgreSQL database using an ODBC driver. Obtaining the ODBC driver: http://www.postgresql.org/ftp/odbc/versions/msi/ In this case we suggest using the following package: psqlodbc_09_00_0310‐x64.zip Installation procedure: Install the downloaded package as directed. qLoader configuration procedure: The native database configuration requires setting the following parameters: 






Type: odbc or custom odbc Server: host address of the server where database server is located Port: where database service is listening in the server (usually 5432) User and Password: of the database service. Database: database where it is located the data we want to load in BIRT Analytics Driver: if the type is “odbc”, we suggest entering the name of the driver as either “PostgreSQL ANSI(x64)” or “PostgreSQL Unicode(x64)” (shown in the 64‐bit ODBC Data Source Administrator in Administrative Tools panel). Leave this option blank if your driver type is “custom odbc”. Connection string: if the type is “custom odbc”, set the connection string as: Driver=Driver_name;Dbq=service_name;Uid=user;Pwd=password 17 of 28 How to load from a MySQL database 4.2
4.2.1 Using native database connection BIRT Analytics is able to connect to a MySQL database using client software: Obtaining the client software: http://dev.mysql.com/downloads/mysql/ In this case we suggest using the package: mysql‐5.6.21‐winx64.zip Installation procedure: 1. Uncompress the zipped file and copy the content into C:\Program Files\mysql 2. Add C:\Program files\MySQL\lib into PATH 3. Restart server qLoader configuration Set the following parameters: 




Type: mysql Server: host address of the server where database server is located Port: where database service is listening in the server (usually 3306) User and Password: of the database service. Database: database containing the data to load into BIRT Analytics 4.2.2 Using ODBC database connection BIRT Analytics is able to connect to a MySQL database using an ODBC driver. Obtaining the ODBC driver: http://dev.mysql.com/downloads/connector/odbc/5.1.html In this case we suggest using this package: mysql‐connector‐odbc‐5.1.13‐winx64.msi 18 of 28 Installation procedure: Download the installation package as instructed. qLoader configuration procedure: Set the following parameters: 






Type: odbc or custom odbc Server: host address of the server where database server is located Port: where database service is listening in the server (usually 3306) User and Password: of the database service. Database: database where it is located the data we want to load in BIRT Analytics Driver: if the type is “odbc”, we suggest entering the name of the driver as: “MySQL ODBC 5.1 Driver” (shown in the 64‐bit ODBC Data Source Administrator in Administrative Tools panel). Leave this option blank if your driver type is: “custom odbc”. Connection string: if the type is “custom odbc”, set the connection string as: Driver=Driver_name;Dbq=service_name;Uid=user;Pwd=password 19 of 28 How to load from an Oracle database 4.3
4.3.1 Using native database connection BIRT Analytics is able to connect to an Oracle database using client software: Obtaining the client software: http://www.oracle.com/technetwork/database/windows/downloads/index‐090165.html In this case we suggest using this package: ODAC1120320_x64.zip Installation procedure 1. Install the downloaded package (includes a path during the process) 2. Restart server qLoader configuration procedure: Set the following parameters: 




Type: oracle Server: host address of the server where database server is located Port: where database service is listening in the server (usually 1521) User and Password: of the database service. Database: database containing the data we want to load into BIRT Analytics 4.3.2 Using ODBC database connection BIRT Analytics is able to connect to an Oracle database using an ODBC driver. Obtaining the ODBC driver: Use the same package as the one used for installing the client (previous section): http://www.oracle.com/technetwork/database/windows/downloads/index‐090165.html In this case we suggest using this package: ODAC1120320_x64.zip 20 of 28 Installation procedure: 1. Install the downloaded package (includes a path during the process) 2. Restart server 3. Modify the tnsnames.ora file, updating the source Oracle information. qLoader configuration procedure: The native database configuration only requires setting the following parameters: 





Type: custom odbc (in Oracle this option is mandatory) Server: host address of the server where database server is located Port: where database service is listening in the server (usually 1521) User and Password: of the database service. Database: database containing the data we want to load into BIRT Analytics Connection string: follows this structure: Driver=Driver_name;Dbq=service_name;Uid=user;Pwd=password 21 of 28 How to load from a MS SQL Server 4.4
4.4.1 Using native database connection BIRT Analytics is able to connect to Microsoft SQL Server database using client software: Obtaning the client software: http://www.microsoft.com/en‐us/download/details.aspx?id=30437 In this case we suggest using this package: SQLServer2008R2SP2‐KB2630458‐x64‐ENU.exe Installation procedure: Install the downloaded package as instructed. qLoader configuration procedure: Set the following parameters: 




Type: mssql Server: host address of the server where database server is located Port: where database service is listening in the server (usually 1433) User and Password: of the database service. Database: database containing the data we want to load into BIRT Analytics 4.4.2 Using ODBC database connection BIRT Analytics is able to connect to a Microsoft SQL Server database using an ODBC driver. Obtaining the ODBC driver: http://www.microsoft.com/en‐us/download/details.aspx?id=36434 In this case we suggest using this package: msodbcsql.msi 22 of 28 Installation procedure: Install the downloaded package as instructed. qLoader configuration procedure: The native database configuration only requires setting the following parameters: 






Type: odbc or custom odbc Server: host address of the server where database server is located Port: where database service is listening in the server (usually 1433) User and Password: of the database service. Database: database where it is located the data we want to load in BIRT Analytics Driver: if the type is “odbc”, we suggest entering the name of the driver as: “SQL Server”. (shown in the 64‐bit ODBC Data Source Administrator in Administrative Tools panel). Leave this option blank if your driver type is: “custom odbc”. Connection string: if the type is “custom odbc”, set the connection string as: Driver=Driver_name;Dbq=service_name;Uid=user;Pwd=password 4.5
Table creation with a database connection Note: This example is done with a PostgreSQL native connection, but the steps described below are exactly the same for any other database connections. 1. In order to create a table using one of the previously defined database connections, click on the “New from database” button in the “Data” tab of the qLoader main screen. 23 of 28 This opens the “Table declaration” window: 2. You must complete the following fields in the “Table declaration” window:  Database: database name on which you want to allocate the table  Name: table name  Action: for complete loads, check “Create”, for incremental loads check “Append”.  Overwrite: check this option if you are planning to execute the project more than once.  Datasource: choose the database connection you defined.  Query: define the SELECT query to extract the data from the database (a Wizard exists to build those SELECT from qLoader, where you can choose the table and the columns you want to load). 24 of 28 3. Then click the “GetColumns” button to see the defined columns and edit their type, width or format, if necessary. When finished, Click “OK”. : 4. Click “OK” to confirm your declaration and return to the main qLoader window. If you have more data to load, repeat the process. 5. When finished you can click on “Save”, in the main screen, to save your project. Before you can run your new project, you first need to synchronize the new repository containing your data with BIRT Analytics. Go to Section 5 of this document to learn how to do this. 25 of 28 5 Synchronize your new tables with BIRT Analytics In order to access the data you have loaded in FastDB engine, you need to add a “post loading” action to synchronize the new repository containing your data with BIRT Analytics. 1. Click on the “Post loading” tab in the qLoader main page to. 2. This opens the “Post loading” screen (see below). Fill in the required fields with the information provided below for a default installation  Application server URL: http://localhost:8110/electronws/services  User: Administrator  Password: PASSWORD 3. Check the “Synchronize” and “Remove cache” boxes Click “Test connection” to be sure that your synchronization process will work. When finished and back in the main screen, click “Save” to save your project and then on “Run” to run it. (Clicking on “Run” automatically opens the log report in the “Running Project” screen.) Note: The first time you run a new project or add new objects, you will need to log in and create access permissions. Go to Section 6 below to find out how to do this. 26 of 28 6 Providing data access privileges for users BIRT Analytics provides a tool that lets you manage who can access your data. It is called BIRT Analytics Admin. (You will need credentials to access this tool). Follow these steps to provide access to your loaded data to your users. 1. Open the BA Admin tool and log in with your credentials. 2. Open the “Access permissions on objects” tab in the “Security role management” panel on the left of the screen. Then click on the “Access permissions on objects” tab. 3. Once in this tab, you have to double click on the “Master users” entry in the users table. 27 of 28 4. The “Updating group” view appears, showing your new data object. In this case, we see a new database called “Flights”. Note: By default the “None” column is checked for it. Check the “All” column box for the object and confirm your action by clicking “Yes” in the pop‐up that appears. 5. Click “OK” and you can logout. Now the data is ready to be used in BIRT Analytics. Note: Synchronizing the data in the repository to BIRT Analytics can also be done using the BIRT Analytics Admin tool. You can remove temporary information from the server here as well. These operations can be done by selecting the corresponding tabs in the left side of the main BA Admin window. More detailed information can be found in our BA qLoader and BA Admin user manuals. 28 of 28