TestApe.com

Comment

Dear Martin,

Thanks for such a wonderful tool which eases time in Unit testing.

Let me introduce myself first. I'm Ananth from Bangalore, India. I'm having a project which needs to be unit tested. I followed the TestApe manual replicated the steps as mentioned. I included the library in Visual Studio 6.0 and also changed the registry settings as mentioned in the manual. When i try to execute, testape.exe crashes. i tried to add the testape.lib towards the end of the Linker settings, but still it doesnt work. I'm not sure if I'm missing something. Please help me out. I'm sending a document with the detailed steps that I followed. Expecting a solution to this ASAP.

Thanks in advance,
Ananth

Name
Spam prevention
Comment
Submit  Cancel

Comment Hi Anath

Thanks for your question - it is always good to know when someone uses the tool. I review your steps - they are correct.

Please tell me : Do testape crash only when you run from within visual studio or also when you run from commandline ?

If you can run from commandline, I suggest you get aquinted with the tool first by using the command line. Eventhough it is possible, the strength of the tool is not visual studio integration. If you get tired of writing commands on the commandline, you can try to add a custom post build step that manually run testape.exe
(e.g. post custom build "testape link obj1.obj obj2.obj testape.lib" ) instead of changing the registry, but I really recommend that you try from commandline until you have your first tests running.

Br,
Martin

Name
Spam prevention
Comment
Submit  Cancel

Comment Thanks for your reply. I'm afraid I wont be using the command line option.
Mine is a big project with over 50 .c files. And i need all these files to be working together. I initially tried to use the commandline option, but i'm not very familiar with that. So, i couldnt achieve much success with it.
But Visual studio integration seemed to solve my problem, though it didnt work as i expected it to. Your manual was very helpful, but I'm not able to fnd out any toubleshooting (despite googling a lot) for TestApe.

actually its not crashing all the time. If i add the lib in the beginning, it crashes. if i add it in between, it throws some error. I'm stuck up. kindly help me out.

Name
Spam prevention
Comment
Submit  Cancel

Comment The testape.lib should be at the end - its important. ( However, it should not crash, but thats another issue ). If it is not in the beginning it throws an error - can you grab that one for me. It can help me figure out what's wrong.

Name
Spam prevention
Comment
Submit  Cancel

Comment Martin,

The "Point 9" in the document that i sent yest is the error that i get. This error is generated only when i add the library somewhere in between or at the end. if i add it at the beginning, it crashes. I'm anyway pasting the error that was generated.

 


--------------------Configuration: coredrv - Win32 Release--------------------


Linking...


LINK : fatal error LNK1181: cannot open input file "ÿ.obj"


   _____________


  ------ OOO ----


          ++


      ,,, ++


     (o_o)**


      (_)**


     **   +


    *+ + .


    ++     ()


    oo  () oo


        oo


Error executing testape.exe.


coredrv.exe - 1 error(s), 0 warning(s)

I have no idea whats that ÿ.obj file. :(

Name
Spam prevention
Comment
Submit  Cancel

Comment

Does webtty support ANSI?

Name
Spam prevention
Comment
Submit  Cancel

Comment With respect to ANSI escape characters, only a few is supported. I think it can be done, but since the entire screen is emulated in a html textarea, it will be difficult to implement for example the concept of a cursor. Colors will most likely be impossible to do in a textarea. I did some experiments long time ago, but they did not work very good though. WebTTY is open source - so if anyone is up for it, they could give it a try.

Name
Spam prevention
Comment
Submit  Cancel

Comment

The following is planned for next release. Your comments are welcome - so far only documentation has been done, so it is still possible to make changes.

Extended mock support

This will be a backward compatible change - no change in macro names or parameters. The main change compared to previous release, is that also functions that are present in the unit can be mocked. This give some extra possibillities.

The mock will typically verify parameters and simulate a return value for the unit. In addition the mock function can choose to call the function it is mocking. TestApe mocks are dynamic. Each test define which functions that are mocked and which mock implementation they use.

Extended mock support changes the following for the EXPECT(function) -->

If the expected function call resolves to a function that is not present in the unit, the framework will report that it is called and return 0 to the unit that is calling it. If the expected function call resolves to a function that is present in the unit, the framework will simply report that it is called and afterwards call the function.

Extended mock support changes the following for the SIMULATE(function, value) -->

If the expected function call resolves to a function that is not present in the unit, the framework will return the value given by the test. If the expected function call resolves to a function that is present in the unit, the framework will not call that function, but instead return the value given by the test.

Extended mock support changes the following for the EXPECT_VALIDATE(function, mock_function) -->

If the mock function is mocking a function that is present in the unit, it is possible, for the mock function to call that function. Calling the mocked function is a possibillity - it is not required. In the sample below imagine that add is actually present in the unit.

int check_add(int left_operand, int right_operand)


{


VALIDATE(left_operand,3);


VALIDATE(right_operand,7);


return add(left_operand, right_operand);


}





void test_addition(void)


{


EXPECT_VALIDATE ( add, check_add );


VALIDATE(calculate(3,7,'+'), 10);


}

Name
Spam prevention
Comment
Submit  Cancel

Comment Implementation is progressing well.

Extended mock functionality have been implemented and is working well for GCC/Linux and GCC/CygWin. Also implemented are commandline support to execute specific tests.

Next beta release will be available June 2009.

Name
Spam prevention
Comment
Submit  Cancel

Comment Beta version has been updated. More information on the new mock functionality as well as known problems on the download page.

Name
Spam prevention
Comment
Submit  Cancel

Comment

There is a change for the forum hosted on this site. The previous phpBB forum is closed for now. All forum threads will be migrated to this simple blog.

In addition some support mails have been migrated to this blog as well, as I believe they are relevant for everyone.

Name
Spam prevention
Comment
Submit  Cancel

Comment This little forum is now also a target for spam, so in order to post here, you need to answer a simple antispam question. The answer is eight - lower, upper or mixed case. If you do not enter 'eight', the post is automatically removed within a couple of minutes.

Name
Spam prevention
Comment
Submit  Cancel

Comment Oh so much spam - from now on all posts will be manualy moderated before they become visible on the forum. Please be patient - serious posts will become visible within a couple of days. Thank you

Name
Spam prevention
Comment
Submit  Cancel

Comment

We have embedded WhiteBox testing requirement for Code coverage, Code profiling, Memory leak and stack/heap exception testing.
We are in process of Automated TOOL selection. We want to know the kind of support available from your tool for our specific set of requirement (like OS, RTOS, IDE, PROCESSOR etc). Please reply back so that we can discuss further.

Name
Spam prevention
Comment
Submit  Cancel

Comment Thanks you for your inquiry. I will try to answer, if it is not adequate, please just ask again.

- The TestApe framework will test embedded code written for any RTOS or any processor as long as the code is written in C.

- TestApe will test the embedded code on a PC. It will generate test code compatible with either Windows or Linux.

- TestApe supports Linux/GCC, Windows/GCC or Window/Visual Studio as compiler options.

- There is no IDE support. TestApe is a command line tool.

In theory TestApe could also support in target testing, but it would require some work and someone to pay in order to make it happen.

Name
Spam prevention
Comment
Submit  Cancel

Comment

I am very interested in the possibilities offered by the tool you developped, TestApe. Unfortunately, I have not been able to test it yet, since I can't compile my tests :)

Could you please tell me what I am doing wrong.
I have this function that I want to test:

------------ 2test.c ------------

int eq(int a) 


{ 


        return (a == 10); 


} 

So i created this test:

------------ testcase.c ------------

#include "testape.h" 





extern int eq(int); 





int main (void) 


{ 


        int ret = eq(9); 


        VALIDATE(ret,1); 


        


        return 0; 


} 

I am under CygWIN to compile and run the tests.

Now, when i want to compile i run:

testape.exe g++ testcase.c 2test.c tesape.a 

(i use g++ instead of gcc, because if i use the latter i get many
errors regarding c++ functions)

The output is:

   _____________ 


  ------ OOO ---- 


          ++ 


      ,,, ++ 


     (o_o)** 


      (_)** 


     **   + 


    *+ + . 


    ++     () 


    oo  () oo 


        oo 





Instrumenting ... _Z20testape_validate_intPcS_ii 

Which means that the linker doesn't find the testape function. Could
you tell me what i am doing wrong?

Name
Spam prevention
Comment
Submit  Cancel

Comment Hi Julien

You need to add extern "C" before including testape.h e.g. like this

extern "C" 


{ 


#include "testape.h" 


} 

The problem is that the g++ compiler requires that the function names in testape.a to be c++ decorated.

Your sample reveals an additioal problem in the framework - you cannot make any validations outside the context of a test. I will fix that in next release, but in the mean time the workaround is only to make validation inside the context of a test.

For example I moved your test code inside test "my_first_test".
I believe the sample below will work.

BTW, what trouble do you see when using gcc ?

Name
Spam prevention
Comment
Submit  Cancel

Comment Thanks for you quick answer, unfortunately it still does not work. The test does not compile, I still have linking errors.

My test file looks like that:

////////////// test.c //////////// 


extern "C" 


{ 


#include "testape.h" 


} 





extern int eq(int); 





void my_first_test(void) 


{ 


      int ret = eq(9); 


      VALIDATE(ret,1); 


} 





int main (void) 


{ 


      EXECUTE(my_first_test); 


      return 0; 


} 


////////////////////////////////// 

Now, when I try to compile with g++:

g++ testape.a func.c test.c 





c:/temp/ccin2Qrh.o:test.c:(.text+0x34): undefined reference to `testape_validate_int' 


c:/temp/ccin2Qrh.o:test.c:(.text+0x74): undefined reference to `testape_execute' 


collect2: ld returned 1 exit status 

And if I use gcc instead (removing the extern "C")

gcc testape.a func.c test.c 





c:/temp/cc0Geaaa.o:test.c:(.text+0x34): undefined reference to `testape_validate_int' 


c:/temp/cc0Geaaa.o:test.c:(.text+0x74): undefined reference to `testape_execute' 


collect2: ld returned 1 exit status 

If I try to use the instrumenter, it "instruments" the missing functions and then exits with errors.

If you have another suggestion on what I should do... I would really like to be able to use testape, the possibilities look quite nice compared to the other unit testing framework have already seen :)

Name
Spam prevention
Comment
Submit  Cancel

Comment The order is important for g++/gcc. The lowest level c/lib/o file should be last on commandline.

Use

> testape g++ func.c test.c testape.a
or
> testape gcc func.c test.c testape.a

I am happy you like the tool.

Name
Spam prevention
Comment
Submit  Cancel

Comment Ok, I finally managed to get the test compiled. Thanks again.

I already have another question though. Is the mechanism for simulating return values of mocked functions implemented?

I use this test code:

int check_Calc(int a) 


{ 


 VALIDATE(a,5); 


 return SIMULATE_VALUE; 


} 





void my_first_test(void) 


{ 


    int ret; 





    EXPECT_SIMULATE(Calc, check_Calc, 1); 


    ret = Calc(5); 


    VALIDATE(ret,1); 


} 








int main (void) 


{ 


    EXECUTE(my_first_test); 


    return 0; 


} 

The test on the return value of Calc (VALIDATE(ret,1)) always fails. I tried using the SIMULATE macro instead of EXPECT_SIMULATE, but I always get a return value equal to 0.

I printf'ed the SIMULATE_VALUE in the validator and it was equal to 0. Am I missing something again? :)

Name
Spam prevention
Comment
Submit  Cancel

Comment I am glad that you made it compile. The SIMULATE_VALUE and related functions are implemented and your sample looks correct.

However, there is a known problem with that, which will be fixed in next release. There is no work-around, besides making several mock functions each returning their own unique value.

Actually I already have it fixed together with a much improved version of the execute functions ( that makes debugging much easier ), but I still need to test it more before making an official release.

I will make a new version available to you within the next couple of days when I think it is ready. Hope you have a few more days to wait

Name
Spam prevention
Comment
Submit  Cancel

Comment As promised - Attached msi or zip file contains a bugfix for the problem about EXPECT_SIMULATE and SIMULATE value that you reported.

Name
Spam prevention
Comment
Submit  Cancel

Comment I'll be glad to test a new version of your tool. I already found several bugs in my project with the current version :)

Name
Spam prevention
Comment
Submit  Cancel

Comment

Hi Martin,

First of all I want to thank you for providing such a great tool on the Internet.

I tried to get it work on my machine but I failed (mostly due to my inexperience with GCC):

Is it possible to build and run it with Cygwin?
I put together a quick and dirty application for using testape (a few lines of C-code, see attachment)

I tried to link it with each of the provided libraries from the win and the Linux packages (testape.lib + testape.a) but it always fails with an "undefined reference to `_testape_validate_int`" error

It would be great to get it work with Cygwin, at least for having a pretext to use a *nix shell at work ;-)

It would be very much appreciated if you provide some very basic example with source code + project files (makefile for Linux, MS-VC project files for Windows). That would be a great help for understanding the inner workings of testape. This may be extremely obvious for you, but not for a first-time user of testape that starts to get involved in unit testing.

Name
Spam prevention
Comment
Submit  Cancel

Comment Thanks for the feedback and for trying out the package - I will help you get it to work with Cygwin. Also, based on your suggestion, I will try to add a "getting started, step by step" chapter in the documentation.

TestApe differs from other unit test tools in the way that it makes all the stub's automatically. It is not important, actually it makes it a bit more complicated in smaller programs, but it is extremely useful when testing applications that operates in a larger "framework".

Anyway - I can see what is wrong and I will fix it within the next couple of days and upload a new version - I'll sent you a mail when it is ready.

Name
Spam prevention
Comment
Submit  Cancel

Comment I have added support for latest CygWin/GCC in the attached zip file. It will be relased officially later when I am done with some other improvements. When using CygWIN/GCC you should link with testape.a from the zip file

I noticed some minor things in your samples -->

1) You should use EXECUTE macro when executing the test, e.g. in test_bed.c EXECUTE(test_load_my_app) instead of just test_load_my_app();

2) You should use the following commandlines when invoking the instrumenter
./testape.exe gcc -L /cygdrive/c/experiments/testape/ testape.a application.c test_bed.c
or
./testape.exe gcc application.c test_bed.c /cygdrive/c/experiments/testape/testape.a

Thanks again for your feedback.

Happy testing
Martin Steen

Name
Spam prevention
Comment
Submit  Cancel

Comment Thanks at lot for your support. I got it running under CygWin. Sorry for the late feedback I was busy migrating to a new PC since on the previous the hard disk began acting up...

Name
Spam prevention
Comment
Submit  Cancel

Comment

I am evaluating different options as a framework for unit testing in an embedded environment. TestApe originally came on my radar as it s homepage specifically mentions embedded development. In reading through the manual however I saw no mention of how TestApe would be used for testing in that environment. Can you point me to documentation that discusses this? My environment is that development will take place on a Windows/cygwin machine for an ARM9 target. Does TestApe support automated unit tests that can be executed on the target? In it still actively being developed? Any planned support for C++?

Thanks for taking the time to respond to my questions. TestApe looks like a pretty cool tool.

Name
Spam prevention
Comment
Submit  Cancel

Comment TestApe is written to support unit testing of embedded software. The way to use it, is to compile portions of the embedded source code ( the unit ) on windows/Linux and use the instrumenter to combine it with the TestApe framework and some tests.

The compilation will generate a text executable that will test the embedded code. The test executable will run on windows/Linux not on the embedded device.

So, this tool cannot be used to test code that interacts inside the embedded environment - instead the code is tested inside the TestApe framework in Linux/Windows. From the embedded code point of view, it will see the embedded environment but the testcase developer determines how this environment behaves when writing the tests.

I understand why some would want to run the tests inside the embedded device it self, but it is not the intention of this tool

The main strength of TestApe compared to other frameworks is that it is very easy to isolate and test only portions of the embedded code - hence it is well suited for very large software projects with a lot of people coorporating.

It has been used to test embedded software in mobile phones. The framework is in its initial state, as you can see by the amount of users in the forum, but it is still being actively developed - and c++ support can be expected in the future.

Name
Spam prevention
Comment
Submit  Cancel