Test Automation G11n Test Suite
- Project name: G11n Automated Testing
- Short description: The goal of this project is to provide g11n test suite using Common Test Infrastructure (CTI) framework and Test Environment Toolkit (TET)
- Responsible person(s) contact: petr.hruska@sun.com
Project Milestones:
- Phase 1 - create G11n basic test suite for OpenSolaris Binary Distro
- Phase 2 - create an extended/regression test suites for OpenSolaris Binary Distro
- Phase 3 - establish a project at OpenSolaris.org, create a mercurial repository there and place a test suites there
- Phase 4 - contribute some tests to testing community at OpenSolaris.org
| Date | Comments |
|---|
| 07/21/2009 | Phase 1 started, evaluation of existing tests |
| 08/13/2009 | Phase 1 review, basic test suite created, currently having 10+ tests |
How to Start
- download TETlite package
- install CTI for TET by standard Solaris package installation tool pkgadd (available also for OpenSolaris) with root privileges:
Failed to execute the [html] macroorg.xwiki.rendering.macro.MacroExecutionException: When using the HTML macro inline, you can only use inline HTML content. Block HTML content (such as tables) cannot be displayed. Try leaving an empty line before and after the HTML macro.
at org.xwiki.rendering.internal.macro.html.HTMLMacro.cleanHTML(HTMLMacro.java:195)
at org.xwiki.rendering.internal.macro.html.HTMLMacro.execute(HTMLMacro.java:145)
at org.xwiki.rendering.internal.macro.html.HTMLMacro.execute(HTMLMacro.java:65)
at org.xwiki.rendering.internal.transformation.macro.MacroTransformation.transformOnce(MacroTransformation.java:184)
at org.xwiki.rendering.internal.transformation.macro.MacroTransformation.transform(MacroTransformation.java:129)
at org.xwiki.rendering.internal.transformation.DefaultTransformationManager.performTransformations(DefaultTransformationManager.java:72)
at com.xpn.xwiki.doc.XWikiDocument.performSyntaxConversion(XWikiDocument.java:7483)
at com.xpn.xwiki.doc.XWikiDocument.performSyntaxConversion(XWikiDocument.java:7432)
at com.xpn.xwiki.doc.XWikiDocument.getRenderedContent(XWikiDocument.java:835)
at com.xpn.xwiki.doc.XWikiDocument.getRenderedContent(XWikiDocument.java:783)
at com.xpn.xwiki.doc.XWikiDocument.getRenderedContent(XWikiDocument.java:878)
at com.xpn.xwiki.api.Document.getRenderedContent(Document.java:545)
at sun.reflect.GeneratedMethodAccessor292.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:389)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:378)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:270)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:252)
at org.apache.velocity.runtime.parser.node.ASTReference.value(ASTReference.java:493)
at org.apache.velocity.runtime.parser.node.ASTExpression.value(ASTExpression.java:71)
at org.apache.velocity.runtime.parser.node.ASTSetDirective.render(ASTSetDirective.java:142)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:106)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
at org.xwiki.velocity.internal.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:196)
at org.xwiki.velocity.internal.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:161)
at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:116)
at com.xpn.xwiki.XWiki.evaluateTemplate(XWiki.java:1895)
at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1833)
at com.xpn.xwiki.api.XWiki.parseTemplate(XWiki.java:860)
at sun.reflect.GeneratedMethodAccessor121.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:389)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:378)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:270)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:252)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:332)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:212)
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:247)
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:106)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
at org.xwiki.velocity.internal.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:196)
at org.xwiki.velocity.internal.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:161)
at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:116)
at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1943)
at com.xpn.xwiki.XWiki.evaluateTemplate(XWiki.java:1856)
at com.xpn.xwiki.web.Utils.parseTemplate(Utils.java:154)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:226)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:117)
at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:129)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:152)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:218)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:354)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
- package is installed to /opt/SUNWstc-tetlite by default
Test Suite Execution
- the test suite execution is required to be done as root
- setup the environment variables:
export TET_ROOT=/opt/SUNWstc-tetlite
export TET_SUITE_ROOT=path_to_test_suite
export CTI_ROOT=/opt/SUNWstc-tetlite/contrib/ctitools
export PATH=$PATH:/opt/SUNWstc-tetlite/contrib/ctitools/bin
- run script
Failed to execute the [html] macroorg.xwiki.rendering.macro.MacroExecutionException: When using the HTML macro inline, you can only use inline HTML content. Block HTML content (such as tables) cannot be displayed. Try leaving an empty line before and after the HTML macro.
at org.xwiki.rendering.internal.macro.html.HTMLMacro.cleanHTML(HTMLMacro.java:195)
at org.xwiki.rendering.internal.macro.html.HTMLMacro.execute(HTMLMacro.java:145)
at org.xwiki.rendering.internal.macro.html.HTMLMacro.execute(HTMLMacro.java:65)
at org.xwiki.rendering.internal.transformation.macro.MacroTransformation.transformOnce(MacroTransformation.java:184)
at org.xwiki.rendering.internal.transformation.macro.MacroTransformation.transform(MacroTransformation.java:129)
at org.xwiki.rendering.internal.transformation.DefaultTransformationManager.performTransformations(DefaultTransformationManager.java:72)
at com.xpn.xwiki.doc.XWikiDocument.performSyntaxConversion(XWikiDocument.java:7483)
at com.xpn.xwiki.doc.XWikiDocument.performSyntaxConversion(XWikiDocument.java:7432)
at com.xpn.xwiki.doc.XWikiDocument.getRenderedContent(XWikiDocument.java:835)
at com.xpn.xwiki.doc.XWikiDocument.getRenderedContent(XWikiDocument.java:783)
at com.xpn.xwiki.doc.XWikiDocument.getRenderedContent(XWikiDocument.java:878)
at com.xpn.xwiki.api.Document.getRenderedContent(Document.java:545)
at sun.reflect.GeneratedMethodAccessor292.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:389)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:378)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:270)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:252)
at org.apache.velocity.runtime.parser.node.ASTReference.value(ASTReference.java:493)
at org.apache.velocity.runtime.parser.node.ASTExpression.value(ASTExpression.java:71)
at org.apache.velocity.runtime.parser.node.ASTSetDirective.render(ASTSetDirective.java:142)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:106)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
at org.xwiki.velocity.internal.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:196)
at org.xwiki.velocity.internal.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:161)
at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:116)
at com.xpn.xwiki.XWiki.evaluateTemplate(XWiki.java:1895)
at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1833)
at com.xpn.xwiki.api.XWiki.parseTemplate(XWiki.java:860)
at sun.reflect.GeneratedMethodAccessor121.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:389)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:378)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:270)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:252)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:332)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:212)
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:247)
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:175)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:106)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:336)
at org.xwiki.velocity.internal.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:196)
at org.xwiki.velocity.internal.DefaultVelocityEngine.evaluate(DefaultVelocityEngine.java:161)
at com.xpn.xwiki.render.XWikiVelocityRenderer.evaluate(XWikiVelocityRenderer.java:116)
at com.xpn.xwiki.XWiki.parseTemplate(XWiki.java:1943)
at com.xpn.xwiki.XWiki.evaluateTemplate(XWiki.java:1856)
at com.xpn.xwiki.web.Utils.parseTemplate(Utils.java:154)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:226)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:117)
at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:129)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:152)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:218)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:354)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Test Suites
G11n Basic Test Suite
| Test ID | Status | Incl. to OpenSolaris Test Suite | Next Steps | Migrated/Created by | Style Review | Functionality Review | Putback Approval |
|---|
logErrorCheck.ksh (OS001-logs,S004-logErrorCheck | ready | yes | copy install logs, warnings should not fail the test | Kristyna | READY | READY | approved, committed |
| S001-init | ready | yes | - | Martin | READY | need corrections | no |
| S002-core_files | ready | yes | transform to g11n regression test | Kristyna | READY | done | approved, committed |
| S003-pkgchk | in progress | yes | modify it for osol, create homepage | Martin | READY | READY | no |
| S005-bounCe | ready | yes | provide also full log | Martin | READY | need corrections | no |
| S006-localedata | in progress | no | merge with S010 | Petko | | | no |
| S007-linkcheck | ready | yes | to be merged with S014 | Kristyna | READY | finished | no |
| S008-lc_check | on hold | no | migration currently not planned | Slavek | | | no |
| S009-broken-links | ready | yes | create homepage, check against g11n packages, include full links in log | Martin | need corrections | need corrections | no |
| S010-validate-locale | ready | yes | S006 to be added, create summary in log, analyze failed status | Petko | READY? | need corrections | no |
| S011-locale-regression-check | completed | yes | create homepage | Petko | READY? | need corrections | no |
| S012-binaryCat | ready | yes | sparc need to be check yet, missing results in report | Kristyna | READY | finished | no |
| S013-mocheck | ready | yes | create list of supported apps | Slavek | need corrections | | no |
| S014-l10n_msgtest | in progress | yes | merge with S007, create summary in log, evaluate if l10n mesg test is possible for firefox | Martin | | | no |
| S015-keyboards | ready | yes | create homepage, check error reporting | Petko | READY? | need corrections | no |
| S016-track_kb_changes | on hold | no | to be evaluated | Petko | | | no |
| S017-duplicate | ready | yes | check against g11n packages | Martin | need corrections | need corrections | no |
| S018-svcs | on hold | no | evaluate services from g11n perspective | Slavek | | | no |
| S019-login-logout | ready | yes | create homepage, ensure session is started the same way as from gdm | Petko | need corrections | | no |
| S020-sorting | ready | yes | add missing golden files, attach full log | Slavek | need corrections | | no |
| S021-x_files | completed | yes | - | Kristyna | READY | finished | no |
| S022-locale_map | completed | yes | - | Martin | READY | READY | no |
| correct_symlink | ready | yes | - | Martin | READY | READY | no |
NOTE: Review collumns should contain these descriptors:
need corrections - there are some issues that should be changed
in progress - meaning is obvious
recheck - corrections have been made and it is necessary to recheck current test (reviewer should recheck the test)
READY - the review was succesfull
Extended Test Suite
| Test ID | Status | Incl. to OpenSolaris Test Suite | Incl. to SXCE test suite | Migrated/Created by |
|---|
| | | | | |
- [Code Review - Test Suites Checklist | TestSuiteCheckList]
Links & Docs
CTI framework is recommended for creating new test suites. See discuss thread
