Xusheng Xiao's Research

Evaluation

2010-02-18T19:27:00.000-08:00

It is good that my evaluation was not so bad.

I got some new ideas that can be integrated into my paper. :)

For the object creation issue, there is another kind of branch - implicit branch. It is created wherever we access an object field. For example, "obj.Name" is trying to access the Name property of obj and it requires that "obj" cannot be null, which is an implicit branch.

If we find DSE fail to satisfy such field accessing result and after all the test executions, the parts of the program that are dominated by the statement of field accessing are not covered, we are sure that there is an object creation issue at that location.

Comment on my paper

2010-01-18T18:20:00.000-08:00

After the discussion about my draft with Dr. Xie, I got lots of excellent feedback. Thus, I want write the most important ones down, so that I won't forget. :)

1. Formally define DSE and find a formal definition of issue. (Refer his Fitness paper)

2. Problem should be to minimize the issues. (Refer his Rostra paper)

3. Related work should add paper of Explaining Failures of Program Analyses

4. The contribution points should have links to research questions and metrics.

Meeting with Justin Hunter

2009-11-30T17:43:00.000-08:00

Today our class had a guest lecturer, Justin Hunter, founder of Hexawise. I expected his lecture to be technical, but it turned out to be a lecture of experience sharing. :) He is not the geek type, but his MBA way of thinking is interesting. When some classmate asked questions about new features of Hexawise, he started to draw a cost and value chart and used it to explain the feature that is low cost and high value would be first considered. This is similar to Dr. Xie' engineering thinking, but more MBA and management style, hehe

After lecture, Justin went to lunch with Rahul and me. Along the way, we discussed what new features of Hexawise could benefit clients more. Since in his PPT, he used google map as an example to show Hexawise, I suggest him integrate Hexawise and Selenium, a web application automatic UI testing framework. But here comes the problem: What should we output to clients? At first, I think of something big and complete: we can use Hexawise to generate the test value and transform into Selenium test scripts; then we can write Ant script to start the execution of these tests in different browsers. To make it more fancy, we can even execute them in parallel: we can set up a central server and execute the scripts there, and the central server will pick up different computers to execute the tests, each computer running a different version of browser, such as firefox in computer A and IE in computer B. If this can be done, then we have a complete solution for client and we can start to sell it!! However, this is not so practical. So I started to think of something more practical and I proposes another small solution: since by using Hexawise, testers need to model the input space for different parameters, we can let the testers or developers from clients to submit their test interface files and use this information to generate the test cases for them. By test interfaces, here I mean JUnit test classes that contains parameterized unit tests for Java, so that the input values generated by Hexawise can be used as the parameters for the PUTs. For C#, it is similar. For web application, we can ask user to upload the web UI page (a html file), so that we can extract the control information and fill the generated input values into different web controls for testing web UI. To know which parameter is mapped to which argument or web control, we let the clients specify the mapping rules, which describe how the parameters are mapped to the arguments in the uploaded test interfaces. This would not increase much burden since in the test design phase, testers have already designed the parameters and values in order to use Hexawise. This sounds like working but not so novel, but in industry practical is more important. :) Rahul also proposed some thoughts to him, but I forgot what exactly they were due to the bad memory caused by not enough sleep....

Though we continuously proposed one and one thoughts about how to improve Hexawise using different techniques, like taint analysis and fault localization, in the end I am not sure whether he would even adopt one thought from us. But one thing I am sure is that, our enthusiastic discussion already made Justin too exciting to eat...

Problem Definition of my project

2009-11-19T23:01:00.000-08:00

After discussion with Dr. Xie, I realized that I lacked of a formal definition of my project, which was why my description was not so convincing. Thus, the top priority thing of my project before next Monday is to come up with a comprehensive and formal definition of the problem I am trying to solve in my project.

Suresh's ICSE 09 paper is a good example to inspire me. My initial idea is to borrow the concepts from shortest path definition from Algorithm Course.

The rough idea is : First I should propose a definition of issues. Then the first condition is to satisfy a constraint of a branch. Then I should propose a definition for human cost and machine cost. When we have a issue reported, we say that machine cost is infinite and we need human's help. For the cost, we always try to minimize Human cost first and then try to minimize Machine cost.

Finally complete the first step of my empirical study......

2009-11-17T20:40:00.000-08:00

After my midterm of algorithm and many other stuffs (taking courses, homework...), finally I force myself to carry out the first step of my empirical study of two subjects: xUnit and MacroScope.

xUnit is the open source project subject used in my course study, so using it is pretty straightforward for my study.

MacroScope is a .NET SQL parser. I got this subject when I was asking Kunal in my lab about how to carry out empirical study. It turned out that this subject was really for the object creation issues as in some methods it reported 20+ object creation issues.

To make my study easier, I use Pex to symbolically execute the public methods of the projects and collect the classes that have non-covered blocks after executions. These classes are good subjects for my study.

The first discovery I find is that for an application, if it is written as a library, it normally would have many object creation issues but not so many uninstrumented method issues. But if it is a user application that develop its functionalities on top of other libraries, then the uninstrumented method issues would be more. I also find that usually in OO application, we need to solve the object creation issue first so that other parts of the program can be reached and uninstrumented method issues would come into play....

Busy weekend

2009-11-15T18:36:00.000-08:00

Since Dr. Xie's feedback for my last draft is that he still did not understand what was the input and output of my approach, I decided to rewrite part of Introduction again to specify the intention of my approach more directly.

For the survey of how many problems of DSE are caused by object creation issue and external method call issue, I only get a good result of xUnit.Assert method. Still need more try on different methods...

For the object creation issue, it seems my previous description is not so clear since I did not think it through. Instead of relying on Pex's report of object creation issues, I tried to report it by myself and camp up with a motivating example that can show the advantage of my new way. Hopefully it sounds reasonable for other people in my group......

For the external method call issue, after I add the symbolic value check, the result turns out to be good. I can get two important external method calls that really affect the coverage out of the 55 reported uninstrumented issues.

A busy but fruitful day......

2009-11-12T20:34:00.000-08:00

This morning I added the example section into my draft and refined the uninstrumented issue example. I also had several interesting finding:
1. I found that now Pex reported testability issue when encountering File operations in the program under test, but I still could get uninstrumented method issues reported for File operation functions.
2. If I use "Console.WriteLine", Pex will not report this uninstrumented method, but if I use "Console.Out.WriteLine" Pex will report it. It seems that Pex currently has some mechanism to deal with the reporting of .NET system libraries.

Based on Dr. Xie's comment, I need to justify my justify my word of "two major problems". I had some rough idea about how to do it, but to make it better, I decided to consult the senior students in my group. :) After asking Suresh, I knew I needed to carry out a survey on some large application (like NHibernate) and put the result in the introduction section. This finally points a direction for me to improve. :)

Around 4:30pm, we had a group discussion about two papers about Input Partition in symbolic execution and Runtime Monitoring of SQL Injection Attack. Though the meeting was a little bit too long (more than 1 hour), we discussed about several ideas and one of them was very promising.

This was really a busy but fruitful day, though I did not get many concrete things done. Hopefully tomorrow morning I can get the survey done and improve my introduction section.

Using T-SQL to assign sequencial number to legacy DB serecords

2009-11-12T19:02:00.002-08:00

To integrate with other system, sometimes we may need to add a column to the old database table and assign some simple sequencial number for every record in that table just like another "ID".

For example, we have a table called "Person" and now we need to add a new column "outsideID" to every record in the table. Using T-SQL, we can do it like this:

T-SQL to generate sequential ID

declare @id uniqueidentifier

declare @name nvarchar(64)

declare @row int

declare @message varchar(80)

set @row=1

DECLARE cur CURSOR FOR

SELECT id, name

FROM Person

OPEN cur

FETCH NEXT FROM cur

INTO @id, @name

WHILE @@FETCH_STATUS = 0

BEGIN

SELECT @message = 'id: ' + CONVERT(varchar(255), @id) + ' name: ' + CONVERT(varchar(255), @name)

PRINT @message

UPDATE Person SET outsideId = @row

WHERE CURRENT OF cur

SELECT @message = 'set outsideId to ' + CONVERT(varchar(255), @row)

PRINT @message

set @row = @row + 1

FETCH NEXT FROM cur

INTO @id, @name

END

CLOSE cur

DEALLOCATE cur

Handling the close event correctly using Outlook Object Model

2009-11-12T19:02:00.000-08:00

In my current project, we need to add a customized toolbar to outlook and attach our event handlers to the close event of the window. Our implementation is like this: when the outlook new mail item window is opened, we attach our event handlers to the mail item and when it is closed, we need to detach the event handlers for preventing memory leak issues. The code we used is like this:

Register Handler to Item Event

   var events = item as ItemEvents_10_Event;
            if (events != null)
            {
                events.Close += OnItemClose;
            }

It is straightforward that we attach to the close event of the item and unregister the handler inside the "OnItemClose" function. However, the handling of the close event is kind of tricky: When a user enters some text in the outlook window and click the close button on the top-right corner or press "Esc", the system will pop up a dialog and ask whether you want to save the changes or not. Before pop up the window, "Close" event is raised and our code gets executed, which unregisters the handlers. Here comes the problem: If user choose "Cancel" in the pop up dialog and return to the mail item, our system works abnormally as the event handlers are gone! To solve this this problem, I consulted many technical articles from the web and finally got the answer: In outlook mail item, there are two "close" event: one is raised when user click close button or press "Esc", the other is a truely close event raised when the outlook window is closed. The first close event is easy to find as it is located on the mailItem object which represents the outlook new mail item. But the second close event is difficult to find as it resides on the inspector object, which represents the window object of the mail item. Thus, the correct code to handle this situation would be:

Register Handler to Inspector Event

   var inspector = this.item.GetInspector as InspectorEvents_10_Event;
            if (inspector != null)
            {
                inspector.Close += OnItemClose;
            }

As this event will only be raised when the window of the mail item is "truely" closed, when user chooses "Yes" or "Not" to close the pop up but not the "Cancel", everything works fine and no memory issues need to be handled.

Some thought of MorphJ--an extension of Java

2009-11-12T19:01:00.003-08:00

Currently, I found an interesting idea of programming language, Class morphing. Class morphing is an abstraction mechanism that allows a class to abstract over the structure of other types. And the main reference morphing language is MorphJ--an extension of Java.

I think MorphJ is so powerful and innovative. In my last project, we adopted NHibernate, an Object Relational Mapping Tool for .NET, for the database development.
With its help, developers need not be aware of the existence of database but just manipulate database records like use normal C# object. To implement this, the "trick" played by NHibernate is that it will create a proxy class for every domain class. When the user read the property of the proxied class, the proxied class will check whether it is necessary to read the data from database. If yes, it will issue a corresponding SQL query and get the requested data. However, in C#, if we map the property to the field of table, we need to declare every property as virtual property, otherwise NHibernate could not override the property due to the language contraints of C#. To change all the property and method into virtual ones is tedious and error-prone. However, after reading your introduction of MorphJ, Ii think this problem could be easily solved by using MorphJ. Assuming that it could support C#, the pseudo code could be like this:

Pseudo Code

class C {

DOMAIN d = new DOMAIN ();

bool hasValue;

< Return *>[Property] for (public Return Property: DOMAIN.properties)
Return Property
{
get
{
       Console.WriteLine("Calling property"+ Property.name + " on "+ DOMAIN.toString());
       if(!hasValue)
       {
             //fetch the data from database
             ......
             hasValue = true;
       }
       return d. Property;
}

}

This new implementation is not only concise, but also easier to maintain and extend.

Make Asynchronous Callback Testable

2009-11-12T19:01:00.001-08:00

Generally, to test an asynchronous callback is not so easy, as you need a way to wait for the execution of the "returned" callback. Some method I found adopts timer and records whether the callback has been executed by using a boolean value. However, this way will make the tests "randomly" fail, since the timer waiting is not stable and suitable for test condition. After consulting some papers from websites, I found an elegant way to test it! Let me explore this idea by starting with a code to be tested:

The code to be tested

public void HandleResult(EvolveResponse response, Exception exception)

{

if (exception == null)

{

var serviceCoverageItemsResponse = (ServiceCoverageItemsResponse)response;

dispatcherWrapper.Invoke(DispatcherPriority.Normal,

new NoArgsDelegate(() =>panel.SetItemsSource(serviceCoverageItemsResponse.ServiceCoverageItems)));

}

else if (!(exception is UnsupportedOfflineOperationException))

{

LOGGER.ErrorFormat("Error occured in handling the operation - {0}", exception);

return;

}

To test this code, we need the help of mock testing framework as it can do the "trick" of replacement of method execution if the expected call meets the conditions. By using method replacement, we can inject the code to store the callback method when the asynchronous method is invoked, and then execute it by hand to simulate the "callback" situation. The callback delegate which should be replaced would be:

Callback Delegate to be replaced

dispatcherWrapper.Invoke(DispatcherPriority.Normal,

new NoArgsDelegate(() =>panel.SetItemsSource(serviceCoverageItemsResponse.ServiceCoverageItems)));

The trick played by Rhino Mock would be:

Trick of replacement of callback delegate

   DispactherCallback callback =
                delegate(DispatcherPriority priority, Delegate createdDelegate)
                    { noArgsDelegate = (NoArgsDelegate) createdDelegate; };

            dispatcher.Expect(x => x.Invoke(DispatcherPriority.Normal, null)).
                IgnoreArguments().Constraints(Is.Equal(DispatcherPriority.Normal),Is.TypeOf(typeof(NoArgsDelegate))).
                Do(callback);

By using this part of code, when the actually code is executed, the "noArgsDelegate" variable will store the actual generated delegate, here is "NoArgsDelegate". So if we invoke this noArgsDelegate later by hand, we could successfully simulate the invocation of "asynchronous callback", which is supposed to be invoked in other thread. The full test code would be:

Complete test code

   [Test]
        public void should_use_dispatcher_to_invoke_the_update_of_service_coverage()
        {
            var panel = MockRepository.GenerateMock<IServiceDetailPanelView>();
            var dispatcher = MockRepository.GenerateMock<IDispatcherWrapper>();
            var eventController = MockRepository.GenerateMock<IEventController>();
            NoArgsDelegate noArgsDelegate = null;
            Guid userId = Guid.NewGuid();
            var handler = new GetServiceCoverageItemsForUserResponseHandler(panel, dispatcher, eventController, userId);
            var items = new []{new ServiceCoverageItemDTO(Guid.NewGuid(),Guid.NewGuid(),10), };
            DispactherCallback callback =
                delegate(DispatcherPriority priority, Delegate createdDelegate)
                    { noArgsDelegate = (NoArgsDelegate) createdDelegate; };

            dispatcher.Expect(x => x.Invoke(DispatcherPriority.Normal, null)).
                IgnoreArguments().Constraints(Is.Equal(DispatcherPriority.Normal),Is.TypeOf(typeof(NoArgsDelegate))).
                Do(callback);
   panel.Expect(x => x.SetItemsSource(items));

            handler.HandleResult(new ServiceCoverageItemsResponse(items), null);
            noArgsDelegate.Invoke();

            dispatcher.VerifyAllExpectations();
            panel.VerifyAllExpectations();
        }

If after the manual invocation of the saved delegate, noArgsDelegate, our verification of "panel.Expect(x => x.SetItemsSource(items));" still could not be meet, then we know that the functional code in the asynchronous callback has bugs!

Hibernate Tips: Efficient Usage of Bag for Association

2009-11-12T19:00:00.001-08:00

More and more people adopted Hibernate as their database development tool, as its implementation of ORM enables developers to manipulate tuples in relational database in object-oriented way. However, using Hibernate is not as easy as it seems like. Let’s have a look at a simple example(NHibernate), adding two new children to the father loaded from hibernate session :

Sample Hibernate Code

var fatherId = Guid.NewGuid();

var father = Session.Load<Person>(fatherId);

var boy = new Person("terry");

var girl = new Person("mary");

father.Children.Add(boy);

father.Children.Add(girl);

Session.Save(father);

And the mapping file for the class Person is as follows:

Sample Mapping File

<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" default-access="field" assembly="Domain.Test" namespace="Domain.Test">

id>

<one-to-many class="Domain.Test.Person, Domain.Test"/>

bag>

class>

hibernate-mapping>

Execute it and it works fine! However, if you attach a sql profiler to your database, you will find that hibernate will first issue delete statement to delete all the children first and then add them back to the father one by one, which means that when you need to add two children, hibernate will issue one delete sql statement and two insert statement! In this case, if the size of father’s children keeps growing, the number of sql issued for a simple add will also increase, which is intolerable.

The reason why hibernate need to delete and add them all back is that the association mapping between father and children is bag and the inverse attribute of it is false! Using bag for association mapping, the domain objects in the collection could be duplicate and without order. However, it is these duplicate and unordered features make hibernate no way to know which domain objects are newly added and which domain objects have been updated or deleted. Thus, the only correct way for hibernate to deal with the change is to remove all the old children and save all the new ones in the collection, which hurts the performance a lot! What’s worse, bag mapping also could not support fetch-join, which could be adopted to resolve N+1 problem. The recommended collection mapping for this usage is to use set instead of bag. Duplicate objects are not allowed in set, so hibernate could differentiate which domain objects are newly added and only issue the necessary add sql statement.

After the analysis, it seems that using bag is really a bad practice in hibernate, but why does it still exist? Actually, if you use bag correctly, it will improve also your performance. The change should be made like this:

Change inverse to true

<one-to-many class="Domain.Test.Person, Domain.Test"/>

bag>

Changing inverse to true means that the modification of father’ children collection will not handled by hibernate. In other words, the association between father and children is maintained by children. Let’s look at the code of adding children to father:

Adding Children to Father

father.Children.Add(boy);
father.Children.Add(girl);

The execution of “father.Children.Add(boy)” will no longer require hibernate to fetch the content of the Children collection from database before the add action, which of course would improve the performance. This efficient way to use bag is actually to make the children colletion of father “read only”: After children data is loaded from database and assigned to father, further change will not be handled by hibernate. Then, if you want to add the children, you should use the assignment of children’s parent property:

Assign Father to Children

boy.Parent = father;

girl.Parent = father;

The summary of using bag is listed as follows:

Always try set for association mapping first
Using bag only when the collection is expected to be “read only” for hibernate
Do remember to set “inverse = true”
Let the objects in the other side to maintain the association

Make UI Composition Testable (Migrate from older blog)

2009-11-12T18:58:00.000-08:00

In General, it is difficult to test the UI composition, like how many buttons and what will the button look like. It may not have so much value that we test what are the exactly height, width and position of the compositions, but it makes sense that we can verify how many controls added to the UI and what are the content of the control. With the help of Rhino Mock and “Factory Pattern”, I can do it like this using C#:

Step 1 – Extract Interface and Introduce a UI Factory

Suppose we are going to create a commandbar for outlook, we have the code like this:

Code To Be Tested

public void AddCommandBar(Outlook outlook)

{

var commandbar = new OutlookCommandBar(…);

//adding buttons to commandbar

commandbar.Add(new Button("1"));

...

commandbar.Add(new Button("100"));

outlook.Add(commandbar);

}

Introduce a factory and Interfaces:

Introduce Interfaces

   ICommandBarFactoryImpl impl = new CommandBarFactoryImpl();
            var commandbarFactory = new CommandBarFactory(impl);
            var commandbar = commandbarFactory.CreateCommandBar();

Step 2 – Mock ICommandBarFactoryImpl for CommandBarFactory:

After step1’s change, we can mock ICommandBarFactoryImpl and test the commandbarFactory. I use Rhino Mock to do this:

Create Mock For FactoryImpl

var mockImpl = MockRepository.GenerateMock< ICommandBarFactoryImpl >();

Then I pass the mock impl to the factory:

Pass mock Impl to Factory

var commandbarFactory = new CommandBarFactory(mockImpl);

Now, I can execute the CreateCommandBar() method and verify what I want!

Step 3 – Move all the add method to the Interface

To test how many buttons are added, we need to verify how many times the Add method has been called. A simple way to test it is to extract all the add methods into parent Interface and transformed them into create methods. In my example, it should be:

ICommandBarFactoryImpl

   public interface ICommandBarFactoryImpl
    {
        CommandBar CreateCommandBar();
        Button CreateButton(string text);
    }

Step 4 – Writing Mock Test to Verify

Everything has been prepared and I can start the testing now!

The code would be:

Final Test Code

   //prepare
            var mockImpl = MockRepository.GenerateMock<ICommandBarFactoryImpl>();
            var commandBar = new CommandBar();
            var button1 = new Button("1");
            ...
            var button100 = new Button("100");

            //expect
            mockImpl.Expect(x => x.CreateCommandBar()).Return(commandBar);
            mockImpl.Expect(x => x.CreateButton("1")).Return(button1);
            ...
            //repeat as many times as you need
            mockImpl.Expect(x => x.CreateButton("100")).Return(button100);

            //execute
            var commandbarFactory = new CommandBarFactory(mockImpl);
            var commandbar = commandbarFactory.CreateCommandBar();

            //verify
            mockImpl.VerifyAllExpections(); //mock verify
            Assert.IsTrue(commandBar.Controls.Contains(button1));
            Assert.IsTrue(commandBar.Controls.Contains(button100));

In this case, we not only could verify how many times the create method has been called, which means how many buttons created, but also verify they all are added to the commandbar!

Decided to start blogging of my research progress

2009-11-12T08:50:00.000-08:00

To keep track of my research progress and new thinkings, I decided to start my blog here!