Writing Integration tests for WSO2 IOT server

Integration testing plays an important role in software testing and it is where we verify that each module of the software functions as it is supposed to be. In WSO2 IOT server, we have written an adequate amount of test cases to cover all the out-of-box functionalities and also provided the extensibility to write customized test cases to cover add-on features. This document will give an introduction on how does the integration tests have been implemented and how can they be extended to cover add-on functionalities.

TestNg has been used as the underlying backbone of the Carbon test automation framework which provides a powerful test execution control mechanism. The framework extensively cater to write test classes with a composition of JMeter and TestNg annotations.

TestNG framework is used to test the functionalities of the IOT server what are mainly known as core CDMF features, such as device enrolment, operations management, notification management etc.

What follows up in this article will guide you through the basics of writing and executing a simple TestNg test case.

Executing testNg test cases

In IOT server source code, you will find all the integration tests residing at the following location of product-iots repository.

product-iots/modules/integration/integration-tests

All the integration test classes are located in src/main subdirectory and the test resources such as testNg.xml file, java keystores and Jmeter scripts are located in the src/resources subdirectory. TestNg engine reads the testng.xml and then executes all the test cases that have been mentioned in the file.

There are several ways to execute the tests. If you want to execute all the test classes at once, you can simply type mvn install in the command line inside the /integration-tests directory. This will execute all the test classes as mentioned in the testng.xml file. If there is only one test class needs to be executed, you can comment out the other test cases from the testng.xml and then run the mvn install command. Optionally, you can execute a particular test class by typing the following command.

mvn surefire:test -Dtest=<name of the test class>

I.e.: mvn surefire:test -Dtest=AndroidEnrollment

Writing a simple test case

Here is a quick overview of some annotations available in TestNG that we will be using in this scenario. You can find a complete list of annotations here.

@BeforeClass — The annotated method will run only once before the first test method in the current class is invoked. In the context of WSO2, you can use this annotation to obtain security tokens for admin services, configure the server and configure any services.
@Test — Marks a class or a method as part of the test. Actual implementation of the test goes here. You may programatically invoke the service and retrieve data and assert them.
@AfterClass — The annotated method will be run only once after all the test methods in the current class have been run. Clear any server configuration made and reset everything to the last configuration to use by the next test in the suit.

Load the integration module to the IDE and create a new package named “SimpleIOTTest” within the tests-integration module. Then create a Java class named SimpleIOTTest and copy the following code. Here, we will be obtaining the licence code for Android Device enrollment and assert whether the response HTTP code is 200.

Then copy the following code snippet in testng.xml resides in resources directory.

Finally, execute the test by simply typing mvn install on the command line inside the tests-integration directory.

Setting up MsSQL and Writing a Simple Java Client

Recently I was happened to setup a MsSql server and connect it to WSO2 G-Reg data-sources. Since it was my first experience with MsSQL server and as it really seemed like a simple walk-through could’ve been very useful for a noob just like I was, it just struck me to come up with write-up. This is nothing more than a step to step guide through setting up a MsSQL server and executing some simple SQL queries from a Java client.

Installing and setting up MsSQL server

First download and install MsSQL server.

When you are installing, make sure to install management tools for SQL server using the feature installation. The management tool will provide you the GUIs that needs to configure MsSQL server.

Once the installation wizard is completed, go to All apps > Microsoft SQL Server 2014* > SQL Server 2014* Management Studio.

*Based on the installation

Then select File > Connect Object Explorer

Select Desktop authentication as the authentication option and click connect. The Desktop authentication option should have prompted at the installation and if you have proceeded with the defaults at installation process it should be available at this stage. Then you will be prompted with the available objects at the root level.

To add a new user, click on security > new > login. Then provide a new login name and a new password. Make sure to select SQL server authentication for the new user since it’ll provide ability to login from, remote programs. Make sure you uncheck “Enforce password policy” and “Enforce password expiration”.

Then open the properties for the new user we just created and assign dbcreator privilege. (logins > root > properties > server roles)

Then select the current server (DESKTOP-A6480CQSQLEXPRESS) and select connect. Select SQL server authentication and provide the username and the password for the user we just created.

Right click on the Databases element under the new connection and click on new database.

Provide the database name as you wish and click OK. Then create a new table on the database as employees and add 2 columns as “name” and “age”. You can create tables simply as right click on the database and create new table. Don’t forget Ctrl+S once you change the schema. (As typical to any windows program) 😉

Example Java Client

Now lets try to write a simple Java class to connect and insert some rows to table, employees.

Create a new Java program on your favourite IDE (mine is IDEA) add the following code in the main class.

Add the correct* MsSQL JDBC driver to the project path to act as the bridge between the program and the MsSQL server. Checkout MsSQL JDBC driver download page here.

*As I encountered, MsSQL Server 2014 and Java 1.7 goes with sqljdbc41.jar

Run the Java client. If everything has gone well, it should output something like following.

PS : In case, it throws a connection refused exception change the MsSQL server port using MsSQL server Configuration Manager. JDBC driver search for the default port 1433 and it needs to be explicitly set in tcp/ip settings. Right click on “Protocols for SQLEXPRESS” and select Properties. Then make sure TCP/IP is enabled and Listen all is true. Also add 1433 as the port under “IP Addresses” tab.

Cheers!

[js-disqus]

Add customized publisher form for a custom RXT

In the previous post I have discussed how to add a custom asset type (Registry Extension) to wso2 governance registry. There can be situations, you need to customize these publisher forms in a way that your organizations requires. Probably adding more labels, changing colours and adding a captcha. Thanks to the maximum extendibility of wso2 governance center, you can do whatever the change your organizations requires. This article will elaborate on changing a particular area of a text field of Governance Publisher related to your custom RXT.

First you need to add your own custom RXT in order to add your own asset type. Read how to add a custome rxt here. For the consistency, you can use following RXT template as the extension.

After adding the rxt, move to the publisher (https://localhost:9443/publisher) and you will see the asset type “Widget” there. Once you click on “add new widget”, you will be redirected to a form like following.

You can do the customization to the form in two versa.

Add a custom.css file for modifying sizes and colours of existing fields.
Write a new template for the form with its own fields.

I will first go through the easy first way and then will jump into the more useful second scenario.

Adding a custom CSS

<custom_rxt_type> should be replaced with the short name of the asset type. (Here it should be “widget”)

Create a new css file such as custom-rxt.css in wso2greg-5.1.0/repository/deployment/server/jaggeryapps/publisher/extensions/assets/<custom_rxt_type>/themes/default/css
Every text-area in create page has its own id. Therefore it can be used to bind a new css rule to the element. Find the id of the text area that needs to be customized. (i.e.: overview_name). Then add your new style for the particular element using css id selector.
e.g.:
```
 #overview_name {
    background-color: green;
    height: 340px;
}
```
Create the following directory structure in wso2greg-5.1.0/repository/deployment/server/jaggeryapps/publisher/extensions/assets/<custom_rxt_type>/themes/default/helpers
Create a new helper file named create_asset.js and add the following content. Add the name of the custom css file to be pushed.
e.g.: o.css.push(‘custom-rxt.css’);

Reload the page. Next time you move to “Add widget” form on publisher you will see the changes have taken place.

Write a new template for the form

Suppose, there is a scenario where you want your organization wants to fill out some details on the form on behalf of the registrants. Or else, you want to check the inputs for a bot using a captcha. The extensible governance center platform provides you the maximum extensibility by adding your own .hbs (handlebars) template to be populated.

Assume that compared to the large palette of input fields in the “Add widget” page, you only need to show the required fields to the end user. The carbon-store platform allows you to override the default form with your own HTML/CSS/JS files.

Create the wso2greg-5.1.0/repository/deployment/server/jaggeryapps/publisher/extensions/assets/widget/themes/default directory structure. This is where we are going to place all the files needs to be used by the particular asset type.
Add 3 directories namely, “css”, “helpers”, “partials” in the path.
Create a file named “create_form.hbs” inside partials folder and enter following snippet there.

If you refer to the source of existing “Add Widget” form, you will notice that elements have been wrapped in such a hierarchy. Input fields for each entry contains the prefix of “overview_” in it’s id and you need to preserve the hierarchy, as we need these input fields to be serialized and sent to the server. ./css and ./js can be used to add more complicated verifications. Also note the form action as well.

On the .hbs context, input fields needs to be identified and mapped to the rxt entries. Therefore, use “overview_<fieldname>” pattern for input names and identifiers.
E.g.:

<input type='text' name='overview_version' id='overview_version' class="form-control validate-required" placeholder="Enter your version here!"/>

Once, you refresh the page you will see your new form in action instead of the default form.

Cheers!

[js-disqus]

WSO2 Governance Registry Adding Custom Artifacts

WSO2 governance registry is the SOA (Service Oriented Architecture) governance model of the wso2 product catalogue. If you are an enterprise that has a lot of SOA artifacts to be stored or governed, you might find wso2 GREG as a handy tool. GReg provides SOA artifacts manipulation in two aspects – as a meta data repository and a governing software.

Out of the box it supports following artifact types to be stored.

SOAP services
REST services
WSDL
WADL
Policies
Schemas
Swaggers

(WSDL, WADL, Policy and Schemas are also known as content type artifacts.)

SOA governance might run beyond the limits of known ready to go artifacts. i.e: Different stages of SOA governance probably need more types of artifacts to be maintained. planning governance stage will probably ask you to store profiles of different supplements and then move them through different life cycle stages. Development artifacts probably run beyond the known descriptive models. But fret not, answer is just a tab ahead in GReg.

Governance registry supports extension of artifacts, life cycles and handlers. This post will describe how to use artifacts extension model of GReg to store more artifacts.

Customizing an existing artifact type

Registry extensions are defined in a file that has the extension .rxt (registry extension). You can edit the rxt file as you wish at Extension tab of management console.

i.e.: Adding another text area named “Department” to REST services.

Browse the Rest services RXT from Management console. (Extensions > Configure > RXT types)
Select View/Edit for Rest Services.
Add following after the version field and save the RXT.

You will see the custom field when you add a new REST service next time and on asset overview page of existing rest service artifacts after version field.
To make the custom field visible on store you need to add it specifically on Store. (To make sure you make it visible for store users). Add following snippet to <GREG_HOME>/repository/deployment/server/jaggeryapps/store/extensions/app/greg-store-defaults/themes/store/partials/asset-attributes.hbs file after version.

Adding a new artifact type

You might probably need to add different type of business governing artifacts considering the scenario. Suppose you need to add an newly arrived inventory to the GReg. (Well, you can use GReg as a inventory management software as well)

Move to Extensions -> Artifact Types of Carbon console and click on add new artifact.

Enter the following xml description for a new inventory.

Then save the artifact descriptor and go to the Governnace Publisher URL. (https://localhost:9443/publisher)

Log in as the carbon user admin (user:admin, pw:admin) and see if our newly added inventory asset type is available. Now you are ready to add new inventory to the repository.

Add new asset and view it on Governance Store. (https://localhost:9443/store)

Please note that you have to make the custom fields available by modifying <GREG_HOME>/repository/deployment/server/jaggeryapps/store/extensions/app/greg-store-defaults/themes/store/partials/asset-attributes.hbs file.

Cheers!

[js-disqus]

Understanding asynchronous JavaScript – Callbacks

Recall Jquery click() function.

Have you ever wondered, why would you want to write the block of the code that needs to be executed after the clicking on the button inside another function?

Welcome to the world of asynchronous programming, callbacks!

Callback is a programming convention rather than a special feature of a language. They are useful when you want your program not to be blocked by a time consuming operation. In this case, finding the exact html element by the id might consume time. Once it is found, the function we pass to the click() function is called back.

See the following code.

Here, the find function takes 2 seconds to finish its operation. It accepts a function as its parameter to be executed once the time consuming operation is done. (Don’t worry about the setTimeout() function. Yes, this too is a calling back function, but for the time being lets assume, all what it does is calling the passed function after 2 seconds). “callback” is just a variable that is used to pass the callback function back and forth. You can change it to whatever the name you want to call it. Even though it’s a time consuming operation, the alert in the main routine “meanwhile I show up” is displayed, regardless the synchronization.

Find function could be something that was defined in the API or somewhere in the program that performs IO operations or network calls. It promises to the main program routine that “I will call you back, once my time consuming operation is done”.

Callbacks with parameters

JS is too awesome to be around that we don’t explicitly need to mention what is being passed to the find function is another function. Be it another function with or without arguments and variables, it treats them the way they like to be treated. We can pass any function alongside any type of variable and a function signature. Inside the caller function (find() function) all we have to do is passing the arguments to the callback properly.

Here, we pass an array of 2 elements to find() function. It takes 2 seconds to find other elements and push them to the elements array. Then the resulting array is passed to the callback function as another parameter and from the major program routine it is caught and displayed.

Rolling in the deep

Here we pass the callback function by reference to the 2nd caller function, process().

Once the results are returned to the firstFunctionCall it sends the results to the process() function to add more elements to the array. The callback function for process() is passed by the reference onMethodsDone.

How to make sure passed function is actually a function?

Once your genius code is behind an API you need to throw proper messages to the user that he has not quit got the idea of callbacks or he has mixed up the function signature. You can use Js typeof operator to make sure the retrieved parameter is a reference to an actual function within the caller function. Try the following code passing a simple integer instead of a function at the firstFunctionCall.

Dealing with states

Your caller function behind the API may be operating a network call and might have gotten a null result. For the sake of modularity you need to treat the failure in a different way so then the user can handle the failure situation inside another function separately. Try following code by passing null instead of [1,2] to the find() function.

Will call you back! 🙂

Cheers!

[js-disqus]

බල්ලොත් එක්ක බෑ – ප්‍රහසන සහ විහිළු

සිව් පදයන් දෙකකි.

චරථ භික්ඛවේ – චරථ භික්ඛවේ
චරථ භික්ඛවේ – චරථ භික්ඛවේ
චරථ භික්ඛවේ – චරථ භික්ඛවේ
චරථ භික්ඛවේ – චරථ භික්ඛවේ

ලේනා ජම්බු ගහේ – ආයේ නගී ආයේ පස්සේ
ලේනා ජම්බු ගහේ – ආයේ නගී ආයේ පස්සේ
ලේනා ජම්බු ගහේ – ආයේ නගී ආයේ පස්සේ
ලේනා ඌ ලේනා!

මුල් සිව් පදය ඇසු විට මට නිකන්ම මතක් වූයේ දෙවැන්නය. පළමුවැන්න සම්මා දිට්ඨි සූත්‍රයේ “චරත භික්ඛවේ චාරිකං – බහුජන හිතාය – බහුජන සුඛාය” යන්නෙහි modified version එකකි. දෙවැන්න නම්, පාසල් කාලයේ බජව් දාන විට ඉගෙන ගත් සිව්පදයකි. (මීට අමතරව මේ තනුවට මැච් වෙන පදයන් තව බොහෝමය. පලකිරීමට සුදුසු තත්වයේ ඇත්තේ මෙය පමණකි.) පළමුවැන්න නම් උගත්තේ රවින්ද්‍ර ආරියරත්නගේ “බල්ලොත් එක්ක බෑ” වේදිකා නාට්‍යයෙනි. මේ ලිපිය චරථ භික්ඛවේ හාස්‍යයට ලක් කිරීම ගැන නම් නොවේ. හාස්‍යෝත්පාදක වස්තු බීජයක් ඇත්නම් සහ එය නාට්‍යයේ ගලා යාමට හානි නොකරයි නම් සන්දර්භය විමසා බලා එය පෙළට ඇතුලත් කිරීම වරදක් ලෙස නොදකිමි. ප්‍රශ්නය ඇත්තේ එවැනි සරල යෙදුම් ගණනාවක් නාට්‍යය පුරා දක්නට ලැබීමය.

මේ තවත් එකකි.

රාජ්‍ය සේවකයා – අන්තිමට යාලු උනේ මොරටුව කැම්පස් එකේ කෙල්ලෙක් එක්ක. වැඩේ ඉතින් ජෝන් බාස්ටම ගියා!

පෞද්ගලික ලේකම් – ඒ මොකෝ? මොරටුව කැම්පස් එකේ කෙල්ලෙක් නම් ඉතින් හොඳයි නේ!

රාජ්‍ය සේවකයා – මොරටුව කැම්පස් එකේ කෙල්ලෝ හොඳයි මළ ගෙවල් වල අඬන්න!

[ගැලරියේ සිනා]

මෙය රවීන්ද්‍රගේ කෘතියේ ලොවෙත් නැති දෙබසක් බවට සැක නැත. මොරටුව කැම්පස් එකේ කෙල්ලන් ගෙන් බහුතරය මළ ගෙවල් වල ඇඩීමට පමණක් සුදුසු බව වෙනම කතාවකි. නමුත් එය නාට්‍යයේ සන්දර්භයට අදාළ නැත. එය ගැලරිය සිනා ගැස්සවීම සඳහා කැම්පස් එක බලා ඇතුලත් කර ඇති දෙබසක් බව පැහැදිලිවම පෙනේ. නාට්‍යය සරල වන්නේ එවැනි සරල යෙදීම් වලිනි.

“අම්මට හු#$%*, සුවපත් වේවා කියපන්..!” කිව්වේ විජය නන්දසිරිය. ලෝකධර්මී නිසා ප්‍රශ්නයක් නැත. ඥානසාර සිවුර දරන් කුණුහරප කියන අතරේ සහ වාසුදේව පාර්ලිමේන්තුවේ පයනු සම්මත කරන කාලයේ ඕනේ පොඩි එකෙක් වට්ටක්කා ද අඳුරයි. නමුත් විජය එසේ කියන්නේ, රංගනය මගින් ප්‍රේක්ෂකයා ඇඳ බැඳ තබාගත් අවස්ථාවක වීම නාට්‍ය රසයට හානි කරන්නකි. එහිදී ඔහු ගැලරිය සිනා ගැස්සවීම සඳහා overacting කරන බව පැහැදිලිවම පෙනී යයි. නාට්‍යයේ මුල් පිටපතින් බැහැරව දෙබස් ඇතුලත් කිරීම නාට්‍ය රසයට පැහැදිලිවම හානි කරන්නකි. තත් කාලයට ගැලපෙන පරිදි වඩාත් updated විහිළු ඇතුලත් කිරීම තිබිය යුතුය. ඒ හැරුන විට, නාට්‍ය ශාලාවේ නම, සන්විධායකයාගේ නම, ළඟ සිටින ප්‍රේක්ෂකයාගේ උස බර දක්වා නාට්‍යයේ පිටපත localize කිරීමක් කල යුතු නැත. මුල් පිටපතින් බැහැරව ඇතුලත් කර ඇතැයි සැක කල හැකි දෙබස් ගණනාවක් දුටිමි. ප්‍රේක්ෂකයාගේ සන්දර්භය ඉලක්ක කර විහිළු සැපයීම් අතින් නම් නාට්‍යය මෙලෝ රහක් නැත. එසේ ඉදිරිපත් කල විහිළු හැරුන විට ඉතිරි බහුතරයක් සාමාන්‍ය ව්‍යවහාරයේ ඒවාය.

රාජ්‍ය සේවකයා – දන්නවද මම කොහොමද ලියුමක් පටන් ගන්නේ කියල, කෙල්ලන්ගේ හිත ඇද…. බැඳ… ගන්න? [ප්‍රේක්ෂකයාගේ කුතුහලය අවුස්සයි]

පෞද්ගලික ලේකම් – නෑ……

රාජ්‍ය සේවකයා – මගේ චූටි පැටියෝ කියල..!

මෙලෝ රහක් නැත. ප්‍රේක්ෂකයාගේ කුතුහලය අවුස්සා කියන්නේ අන්තිම සාමාන්‍ය දෙයකි. එතරම් පූර්විකාවක් එම වදනට ලබා නොදිය යුතුව තිබුනි.

රාජ්‍ය සේවකයා – දවසක් මම මගේ චූටි පැටියෝ කියල පටන් ගත්ත ලියුමක් දුන්නනේ ලොරි බාගයක් සයිස් කෙල්ලෙක්ට. [සිනා] ඉතින් ඒකිගේ අයිය ආවා අපිට ගහන්න. අපි හතර දෙනාම සූරයෝ බලල හිටිය වෙලේ ආවේ…

අන්තිම සරලය. එකක් මෙහි ඇති සාමාන්‍යත්වයි. අනික මට සිතුනේ නම්, ලොරි බාගයක් සයිස් කෙල්ලෙක්ට එවැනි ලියුමක් ලැබුනේ නම් ඒකිගේ අයියා මුට ගැසීමට පැමිණීමෙන් වැඩක් නැත. කර යුතුව තිබුනේ වහාම බන්දා දීම වගේ එකක් බවයි.

කෙසේ නමුත් රංගනය අතින් වඩාත් හාස්‍යයක් ජනනය කිරීමේ හැකියාවක් ඔවුන් හට තිබුණි. පෞද්ගලික ලේකම්ගේ moon walk එක නම් පට්ටම සිරාය. පෞද්ගලික ලේකම්, පොඩි හාමුදුරුවන් ලෙස වෙස් වලාගෙන සන්සුන් ලෙස වැඩම කිරීම නිම වන්නේ moon walk එකකිනි. ඒ රාජ්‍ය සේවකයා හාමුදුරුවන්ගේ සන්සුන් කම දැක “අනේ වාසනාවන්” කිව්වා පමණක්ම වීම හාස්‍යය තීව්ර කරවන්නකි. එයද සමහරවිට ප්‍රේක්ෂක සන්දර්භය ඉලක්ක කර කල විහිළුවක් වීමට පුළුවන. එසේ නම්, ඒ සාර්ථක යෙදීමකි.

තේමාව අතින් ගත් කල නම් එය ලංකාවේ ජාරජීර්ණ සහ කටුසු දේශපාලනයයි. රාජ්‍ය සේවකයන් කුජීත වී ඇති ආකාරයයි. ශාසනය කෙරෙහි ඇති ආලය දුරුවී විටෙක විහිලුවක්ද, විටෙක අබුද්ධිමත් චීවරධාරින්ගේ තෝතැන්නක්ද වී ඇති ආකාරයි. එකී මෙකී බොහෝ දෘෂ්ටීන් ඔස්සේ ගොඩනැගුනු බල්ලොත් එක්ක බෑ හි තේමාව අත්‍යන්තයෙන්ම ඒ ආකාරයේ විවේචනයක් උවමනා කර ඇත්තකි. රුපියක් දහසට විකිනෙන ගණිකාවද පෙන්නුම් කරන්නේ එකී මෙකී දේශපාලන අරගල තුල සමාජයේ තවත් පැතිකඩකි. වේශ නිරූපන ශිල්පියාගේ භූමිකාව නම් පැහැදිලි නැත. සමහර විට, සාමාන්‍ය පන්තියේ තරුණයෙක් වීමට පුළුවන. කෙසේ නමුත්, නාට්‍යයේ අරමුණ බොහෝ වෙලාවක් යනතුරු අපැහැදිලිය. යම් යම් විටෙක සම්පූර්ණයෙන්ම track පැන්න බව නම් පෙනී ගිය කරුණකි. රාජ්‍ය සේවකයාගේ බයිලා ගැසීම ඕනෙවට වඩා උලුප්පා දක්වා තිබුණි. “අපිත් රාජ්‍ය සේවකයෝ” යන වදන් පෙළ ඇසීමත් අන්තීමට එපා වෙන ගානටම ඔද්දල් කරනු ලැබුවේය. ධන ලකුණු නොතිබුණාම නොවේ. සිදුවූයේ බහුතරයක් තිබුන ඕනෙවට වඩා බයිලා ගැසීම නිසා ඒවා යටපත් වීමය. නාට්‍යයේ ධාරාව කිසියම් ස්ථාවරත්වයකට පැමිණෙන්නේ හාමුදුරු වෙස් ගන්නා ජවනිකාව ආසන්නයෙනි. අවසන් ජවනිකාවේද, ජයග්‍රහණයෙන් පසුව දේශපාලකයා, ජනතාව පිස්තෝලයක් පෙන්වා පළවා හැරීම ඕනෙවට වඩා ඍජු, පොල්ලෙන් ගසන්නාක් වැනි ඉදිරිපත් කිරීමකි. එය එතෙක් වෙලා සංයමයෙන් යුතුව ගොඩනගාගෙන ආ භාවයන් සුනු විසුණු කිරීමකි. එය ඊට වඩා ආනතියක් සහිතව කල යුතුව තිබුණකි.

සමස්තයක් ලෙස හැඟෙන්නේ නම්, අනවශ්‍ය රඟපෑම් වලින් නාට්‍යයේ මුල් තේමාව සහ සන්දර්භය යටපත් කර ඇති බවයි. යම් යම් තේමාවන් අතින් සාර්ථක බවක් අත්කර ගත්තද, ප්‍රධාන තේමාව නාට්‍යය තුලදී අසාර්ථකත්වයට පැමිණ ඇතැයි සිතේ. සමහර විටෙක ප්‍රහසනයකටත් වඩා විහිළු සැපයීම අතින් නම් නාට්‍යය ඉදිරියෙන් තිබුණි.

State Estimation with Kalman Filter

Kalman Filter (aka linear quadratic estimation (LQE)) is an algorithm which can be used to estimate precise states of a moving object by feeding a series of noisy sensor inputs over time.

The Kalman Filter and its derivatives namely, “Extended Kalman Filter (EKF)” and “Unscented Kalman Filter” are highly reputed in the field of information processing. The most famous early use of the Kalman filter was in the Apollo navigation computer that took Neil Armstrong to the moon, and (most importantly) brought him back. Today, Kalman filters are at work in every satellite navigation device, every smart phone, and many computer games.

The Kalman filter requires a dynamic model of the system to predict next states of the system. The dynamic model can be motion laws or other equations that has ability to calculate the next state with use of correct coordinates.

Kalman Filter performs on two main operations. They are,

Estimating the next state of the machine
Correcting the estimated state with actual measurements

The following picture shows estimated location (after the correction) against measured location and the true location.

Modeling Kalman Filter

To model the scenario with Kalman filter, it requires to model two equations.

Process model (State Equation)

xt is the state vector containing the terms of interest for the system (e.g., position, velocity, heading) at time t
ut is the vector containing any control inputs (steering angle, throttle setting, braking force)
Ft is the state transition matrix which applies the effect of each system state parameter at time t-1 on the system state at time t (e.g., the position and velocity at time t-1 both affect the position at time t)
Bt is the control input matrix which applies the effect of each control input parameter in the vector ut on the state vector (e.g., applies the effect of the throttle setting on the system velocity and position)
w(t) is the vector containing the process noise terms (unknown) for each parameter in the state vector. The process noise is assumed to be drawn from a zero mean multivariate normal distribution with covariance given by the covariance matrix Q(t ).

Measurement model (measurement equation)

zt is the vector of measurements
Ht is the transformation matrix that maps the state vector parameters into the measurement domain
vt is the vector containing the measurement noise terms (known) for each observation in the measurement vector. Like the process noise, the measurement noise is assumed to be zero mean Gaussian white noise with covariance Rt.

For a simpler example let’s consider a scenario of a car, moving on X- axis with a constant acceleration (a). Suppose car emits its X coordinate periodically. But due to the measurement noises, the measurement can be vary from the actual location.

State vector contains the location and the velocity of the car over the X axis.

Using motion equations,

Now this is the state equation in matrix form using 1 and 2,

Suppose we are reading speed and the location and the speed with white measurement noises. This is the measurement model in matrix form.

With 3 and 4, we can estimate the states of the machine. If you are using a Kalman implementation library, it will do the rest of the calculation given 3 and 4 models. i.e Apache Commons Math Kalman Filter implementation.

Kalman Filter equations

Kalman Filter maintains the estimates of the state and the error covariance matrix of the state estimation.

Notations:

X(t|t) — Estimate of x(t) given measurements z(t) and z (t-1),….
X(t+1|t) — Estimate of x(t+1) given measurements z(t) and z (t-1),…
P(t|t) — Covariance of X(t) given z(t), z(t-1),…
P(t+1|t) — Covariance of X(t+1) given z(t), z(t-1),…

State Estimation

Known are x(t|t), u(t ), P(t|t) and the new measurement z(t+1).

Time Update

State Prediction x(t +1|t )= F(t ) x(t|t) + G(t ) u(t )
Measurement Prediction: z(t+1|t)= H(t)x(t+1|t )

Measurement Update

Measurement Residual: w(t+1)= z(t+1) — z(t+1|t)
Updated State Estimate: x(t+1|t+1)= x(t+1|t) +W(t+1)w(t+1)

Where W(t+1) is called Kalman Gain in state covariance estimation.

State covariance Estimation

State prediction covariance: P(t +1|t ) = F(t)P(t|t )F(t )’+Q(t )
Measurement prediction covariance: S(t +1) = H(t +1)P(t +1|t)H(t +1)’+R(t +1)
Filter Gain: W(t +1) = P(t +1|t )H(t +1)’ S(t +1)-1
Updated state covariance: P(t +1|t +1) = P(t +1|t ) — W(t +1)S(t +1)W(t +1)’

Hopefully, I’ll cover the implementation of a Kalman filter using Apache Math library in my next post!

Cheers!

Using Hg Mercurial behind a proxy

You might have faced to download gigabytes of repositories with Hg. In my case, a big portion of Mozilla repositories depends on it and wanted to download that huge repos from the university network. University bandwidths are mostly underused 😉

Hg uses hgrc file to load configurations. (Per repository) You can go to your repository, .hg/hgrc and add following with your configurations.

[http_proxy]
host = your.proxy.host:8000
passwd = password
user = username

In case you are using hg from a preloaded instance to clone repositories to another location, you can simply add your configurations as a prefix to your command.

hg --config http_proxy.host=[proxy_ip]:[proxy_port] --config http_proxy.user=[username] --config http_proxy.passwd=[password] push

Make sure if you have special characters(@,%,,.) in your username or password you use the suitable character representation for it. ie : use %40 instead of @ symbol. This will also help in finding special character representation.

Cheers!

Writing Axis2 Handlers

Interfering the message flow

Axis2 handler is the smallest invocation unit of the axis2 engine, which has ability to intercept into the message flow during the runtime and operate read/write operations on an incoming or outgoing message. i.e: if you need to route messages coming with specific attributes to a different end point or count the number of messages with that attribute you can simply make use of axis2 handlers. Since, handlers has that ability of intercepting to message flow and read/write to message context, inherently they are capable of suspending the message flow.

Axis2 handlers give you full authority over SOAP messages travel through. As a result of that super control, handlers can add additional headers to SOAP message, add body content and read the message header or the body.

Specially, when implementing ws-reliableMessaging handlers can be used to control the flow of the message. if the message supposed to be delivered second, arrives first it can be hold till the first message arrives and then it can be send to execution.

Inbuilt axis2 handlers

Axis2 comes with a list of inbuilt handlers to implement ws-* and other functionalities like parsing headers.

Writing an axis2 Handler

To write a handler, you either have to extend the AbstractHandler class or implement the Handler interface.

[java]/**
* Created by malintha on 9/25/14.
*/
public class MyHandler extends AbstractHandler {

private static Log audit = LogFactory.getLog(MyHandler.class);
@Override
public InvocationResponse invoke(MessageContext messageContext) throws AxisFault {
SOAPEnvelope mes = messageContext.getEnvelope();
SOAPHeader mesh = mes.getHeader();
SOAPBody mesb = mes.getBody();
OMElement bodyChild = mesb.getFirstElement();
//TODO statements
}
//This return statement defines what to be done with the message flow after this handler
//execution is finished.
return InvocationResponse.CONTINUE;

}[/java]

The return statement of the

invoke()

method defines what has to be done with the message flow after the execution of current handler is finished.

Continue: The handler thinks that the message is ready to go forward.
Suspend: The handler thinks that the message cannot be sent forward since some conditions are not satisfied; so the execution is suspended.
Abort: The handler thinks that there is something wrong with the message, and cannot therefore allow the message to go forward.

In most cases, handlers will return

InvocationResponse.CONTINUE

as the return value.

An axis2 phase consists of several handlers and after each handler calls InvocationResponse.CONTINUE after the execution, the message is immediately passed to the next handler

Packaging the handler

Easiest way of packaging a handler is shipping it with an axis2 module. By that way, each handler can act independently and number of handlers can be shipped.

A module is a collection of handlers along with its configurations. We can define more than one handlers within a module.

Create the following directory structure in your project.

LoggingModule

-Resources
 -META-INF
  -module.xml
-src
 -LoggingModule.java
 -LoggingHandler.java

LoggingModule.java No implementation in this class at the moment.

LoggingHandler.java

“module.xml” contains the deployment configurations for a particular module. It contains details such as the Implementation class of the module (in this example it is the “LoggingModule” class and various handlers that will run in different phases). The “module.xml” for the logging module will be as follows:

InFlow – Represents the handler chain that will run when a message is coming in.
OutFlow – Represents the handler chain that will run when the message is going out.
OutFaultFlow – Represents the handler chain that will run when there is a fault, and the fault is going out.
InFaultFlow – Represents the handler chain that will run when there is a fault, and the fault is coming in.

Then you can modify the axis2.xml in order to introduce LoggingModule to the axis2 engine. Here we have defined a custome phase named LoggingPhase here.

services.xml

After the module is built, you may place the .jar or .mar (rename the jar file) and keep in your library dir WSO2, (repository/components/lib).

Writing WSO2 IS Integration Tests

Integration testing is a phase of software testing cycle in which we verify that the software/module/component functions as it is supposed to be. Generally in any platform integration testing shares some mutual behaviours. We write each test class against a particular function of the module. After invoking each function programmatically the retrieved results may asserted against preferred results. If the results are differed from the expected value, the assert function will fail the test.

Setting the background for WSO2 integration tests

If you have downloaded the WSO2 platform and turing you can see the test framework packed in turing/platform-integration. The test framework contains classes that needs to invoke wso2 services programatically, admin service clients, parent test classes for each product and assert functions. Therefore, prior to writing any tests you need to build the test framework. This will put up all necessary jars to your local .m2 repository. If you can find this directory listing in your .m2 repo, you may not need to build it again.

[shell].m2/repository/org/wso2/carbon/automation[/shell]

[shell]org.wso2.carbon.automation.api
org.wso2.carbon.automation.core
org.wso2.carbon.automation.extensions
org.wso2.carbon.automation.tools.jmeter
org.wso2.carbon.automation.utils
test-automation-framework[/shell]

If not, you can build it using mvn install command at turing/platform-integration/test-automation-framework/4.2.X.

[shell]~/WSO2-Carbon/turing/platform-integration/test-automation-framework/4.2.8$ mvn install[/shell]

Running test cases

In the WSO2 repository you can find test cases written for each product within it’s module/ directory.

In this case you can find tests for WSO2 Identity server at,

[shell]turing/products/is/5.1.0/modules/integration/tests[/shell]

src/tests/java/ contains all the test classes while src/test/resources contains all the test resources.

All the available tests are manifested in a xml file located at src/test/resources/testng.xml

WSO2 uses testng as underlying test framework and read more about Testng maven testng.xml here.

<classes> contains all available test cases which can be executed by testng framework. (Notice all test cases ends up with the postfix “TestCase”)

[xml] <class name="org.wso2.carbon.identity.tests.application.mgt.ApplicationManagementTestCase"/>
<class name="org.wso2.carbon.identity.tests.saml.SSOConfigServiceTestCase"/>
<class name="org.wso2.carbon.identity.tests.sts.SAML2TokenRenweTestCase"/>[/xml]

You can run all available tests by running mvn install command at turing/products/is/5.1.0/modules/integration/tests/. This will execute all the tests mentioned in the testng.xml. If you need to omit a testcase being executed simply comment the particular line at testng.xml.

Writing your own testcase

Load the integration module to your IDE. (IDEA will prefer loading the whole bunch of turing platform if you are not preferred to load libraries explicitly while eclipse satisfies with the necessary unit like integration module)

Create a new package named “Mytest” at integration/tests/src/test/java/org/wso2/carbon/identity/tests
Create a new java class named MyIntegrationTestCase.java

Now extend the MyIntegrationTestCase from ISIntegrationTest which provides an abstraction of IS related details to your test. You can use testng annotations to indicate testng executor at which point of the test the particular method needs to executed. Most commonly using annotations are,

@BeforeClass – The annotated method will be run only once before the first test method in the current class is invoked. In the context of WSO2 IS, you can use this annotation to secure admin services, configure the server and configure any services.
@Test – Marks a class or a method as part of the test. Actual implementation of the test goes here. You may programatically invoke the service and retrieve data and assert them.
@AfterClass – The annotated method will be run only once after all the test methods in the current class have been run. Clear any server configuration made and reset everything to the last configuration to use by the next test in the suit.

Here we will write a single test method under @Test annotation to check the operating system and assert it.

[java]
package org.wso2.carbon.identity.tests.MyTest;

import junit.framework.Assert;

import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import org.wso2.carbon.automation.core.utils.serverutils.ServerConfigurationManager;
import org.wso2.carbon.identity.tests.ISIntegrationTest;
import org.wso2.carbon.utils.CarbonUtils;

public class MyIntegrationTestCase extends ISIntegrationTest {

String carbonHome = CarbonUtils.getCarbonHome();
ServerConfigurationManager scm;

@BeforeClass(alwaysRun = true)
public void testInit() throws Exception {
super.init(0);
scm = new ServerConfigurationManager(isServer.getBackEndUrl());
log.info("Preconfigurations takes place here");
}

@Test(alwaysRun = true, description = "runStsClient", priority = 1)
public void runStsClient() {
String OS = System.getProperty("os.name").toLowerCase();
try {
if (OS.contains("windows")) {
log.info("Operating System is Windows");
Assert.assertEquals(OS, "windows");
} else if (OS.contains("linux")) {
log.info("Operating system is not windows.");
Assert.assertEquals(OS, "linux");
}
} catch (Exception e) {
e.printStackTrace();
}
}

@AfterClass(alwaysRun = true)
public void endTest() {
try {
scm.restoreToLastConfiguration();
} catch (Exception e) {
e.printStackTrace();
}
}
}
[/java]

Now, you need to add the test in testng.xml file.

Place the following line just before the </classes> tag.

[xml]<class name="org.wso2.carbon.identity.tests.MyTest.MyIntegrationTestCase"/>[/xml]

Run the test

Save the files and run mvn install to run tests as mentioned above.
Clue : you can comment out other test cases to see your testcase getting executed soon.

Cheers!