domingo, mayo 20, 2007

Construcción de software, Fábricas de Software

Mientras continúo pasando a un lugar más accesible mis notas, van apareciendo materiales de interés olvidados. La cita recordada aquí refiere a un artículo en The Server Side de Griffin Caprio, que, en el marco de la "popularización" de las herramientas de Microsoft orientadas a software factories, ofrece una visión más imparcial y generalizadora del problema que la que otras declaran. Lo más interesante del artículo:

Moving from Craftsmanship to Industrialization
All too often, highly skilled application developers and architects have to use their time for low-level, implementation level tasks. Usually, junior developers are not able to complete such tasks because of lack of appropriate domain knowledge, requiring the senior developer to mentor the junior developer. This fosters not only knowledge transfer, but also an introduction to the complexities of the current development environment.
Since developers are always involved at some stage of development, very little time is spent in making development more efficient, especially low-level implementation details. This method of development resembles early goods based industries, where single workers create custom solutions, tailored to each individual requirement. Think early tailors or shoe cobblers.
This craftsmanship approach to software development does not scale very well. The lack of quality senior developers creates a mentoring environment, where specialized knowledge must be transferred, similar to an apprenticeship. Since there is such a hands-on approach required, each part of the project need to be created, most of the time, by hand. This often leads to higher quality, but also leads to performance and efficient issues.
Migrating from a craftsmanship-based industry to a more industrial-based industry has been the path of progression for many more mature industries. If this is the end result for so many other industries, why is software development still based on small groups of highly specialized craftsmen?
Most people within the IT industry will agree that a form of standardization and modularization is the key to enabling the kind of reuse required for efficient industrialization of software development. What they don’t agree on is the means to which this standardization and modularization is achieved. Software Factories aim to address this effort by prescribing a process by which software can be modularized into reusable assets.

Caprio se refiere a los tres componentes de las Factorías de Software (en el sentido SF de Greenfield): Modelos y Patrones, Lenguajes específicos de Dominio (DSL), y Lineas de Productos (SPL). Siendo un artículo temprano (marzo 2005), aún no insistía en lo que luego se convirtiera en una oposición de tipo comercial entre la iniciativa de la OMG (Model Driven Development, MDA) y Microsoft (Software Factories, SF, Domain Specific Languages, DSL):
Sobre el papel de los modelos, Caprio dice:

“using models to capture high level information, usually expressed informally, and to automate its implementation, either by compiling models to produce executables, or by using them to facilitate the manual development of executables”
The importance of models comes from their ability to keep a consistent representation of concepts within a project. For example, while it’s easy to draw and manipulate a person object in a model, it’s much more difficult to manipulate the same person object at a lower level, because the person object could be represented by class files, tables, and columns in a database.

Dejo de lado las palabras sobre DSLs, que para 2005 eran sólo una expresión de deseos. En cuanto a las líneas de producto de software, indica:

Software Product Lines are entire subsets of components that can be configured, assembled, and packaged to provide a fairly complete product. The resulting product should only require customization from a developer for the highly specialized aspects of the project.
Perhaps the largest component of a Software Factory, Software Product Lines not only provide the greatest value, but also require the greatest investment. Software must be carefully partitioned into distinct and reusable components and those components must readily fit together in a coherent manner. Configuration is the key to Software Product Lines, as projects must be able to pick and choose which components they want to utilize, and then generate an application off of that configuration.

Pero lo más interesante de este artículo temprano, es que se refiere no solo a las iniciativas de Microsoft, sino a las de Sun. De ellas se puede hablar un poco más, y así abrir un poco más las ideas sobre el problema. Así presenta el tema Caprio:

While all of the promises of Software Factories sound appealing, many companies have tried to provide the tools and components, only to fail under the load of inflexible tools or proprietary formats.
All of this is about to change. Big name companies like Microsoft and Sun are getting ready to release many of the components necessary for building and assembling a Software Factory within an organization. With the release of Visual Studio 2005, Microsoft will unveil several add-ins and plug-ins that enable the creation of not only Domain Specific Languages, but also the integration of those languages with the IDE itself. This will allow developers to manipulate and use the language from within the Visual Studio.NET IDE. Not to be outdone, Sun Microsystems is working on its own implementation of Software Factory technology, simply named ‘Project Ace’. Although, very little details of ‘Project Ace’ are available, developers shouldn’t expect Sun to let Microsoft provide .NET tools, without answering with a comparable set of tools for Java.

Bien, qué hay con el proyecto de Sun, ACE? En la presentación de Sun:

The Ace Project at Sun Labs has developed a way to fundamentally simplify the process of creating modern, web-enabled business applications.

  • Applications are specified in the Ace language, DASL§, which is a higher level architectural programming language. This language is the gem at the heart of Ace.
The DASL language abstracts out complexities, such as persistence and distribution. Using DASL, the application writer concentrates on domain details specific to the application's purpose, instead of worrying about the application's deployment architecture, middleware APIs, remote object invocations, and other details of the implementation "stack". The DASL language defines precise application and business logic and semantics, including application-level transactions.
  • DASL is based on published specifications and therefore can be considered open.
  • Ace is not a tool, it is a language which has a reference implementation that includes an ease-of-use tool.
  • Although the DASL language is compiled in the reference implementation, it can also be interpreted.
  • The generators which come with Ace can be customized.
  • The next version of the reference implementation will have a pluggable Ace generator API.
  • DASL can call Java code.
  • Ace doesn't compete with Java, it works at a higher level and generates pure Java code.
  • The distribution details are implemented automatically at the time the application is compiled and deployed, based on the well defined semantics of the DASL language.
  • There is no Ace runtime.
  • The Ace approach results in tremendous cost and time savings when creating a J2EE Application.
  • As new architectures beyond J2EE evolve, new code generators can be written to support them without modifying the DASL language, and without rewriting applications written in it.

The preliminary results of this new way of building business applications have been remarkable. We have found that in typical web applications as they are written today, roughly 5 to 10% of the code has to do with domain logic, and roughly 90 to 95% has to do with distribution. Using our approach, there is a huge reduction in the number of lines of code required to specify these applications, and a corresponding huge reduction in the time it takes to write and debug the applications.

For example, the Java J2EE Pet Store application can be specified using about 1000 lines of Ace code, as compared to 14,000 lines of code in the native J2EE implementation. In fact, of those 1000 lines of Ace code, 750 of them can be created graphically via UML diagrams that are part of our GUI development tool, so the entire Pet Store is actually specified in Ace by writing only 250 lines of traditional "code".
(...)
The Ace project team believes that application programming languages must evolve to handle the distribution automatically. Just as early programmers who wrote in assembly language had to worry about register allocation, stack protocols for invoking library functions, and other details of the machine on which the applications were run, today's programmers worry about passing data and objects back and forth between various computers, services, languages, and data sources. We have defined DASL as the first of a new breed of higher level languages that abstract out the distribution details and handle them efficiently by capturing the global semantics of the application.
(...)
The benefits of our approach to application writing include:
  • Huge reduction in the time it takes to write and debug applications (many fewer lines of code).
  • Ability to retarget existing applications to new deployment architectures without having to change the application specification. Retargeting is crucial for several reasons:
    • Applications initially written on a small scale (smaller number of simultaneous users) often must be redeployed with additional tiers if they become popular, e.g., the customer base grows.
    • As hardware and technology advance, existing applications must often be rewritten for the new technology.
  • Automatic verification of correctness of application-level transaction semantics.
  • Global optimization of communication between the browser, web server, proxy server, application server, and database server tiers. While such optimization is possible today, it is so tedious and time-consuming that it is often not done.
Si bien el proyecto ACE no parece estar activo, de todas formas estudiar sus objetivos contribuye a tener una idea más amplia del problema a resolver.
Sobre su lenguaje DASL, en Wikipedia.

5 comentarios:

Jorge Ubeda dijo...

Gabriel, gracias por advertirme...debe haber inhabilitado los comentarios sin darme cuenta. Sin embargo, son bienvenidos...Activado de nuevo. Tendré que revisar todos los posts. Es un problema de los defaults de las opciones de posteo.

Gabriel C. dijo...

Me imagine que era por accidente :-)
Todavia estoy trantado de entrarle al tema de DSL y MDA.
Igual creo que la analogia "artesania-industrializacion" es media engañosa:
en particular, con los sastres y zapateros, los nivles de complejidad
no parcen ser muy comparables, tal vez los fabricantes de relojes y
aparatos a cuerda tuvieran un nivel de complejidad similar aunque el
software es mucho mas critico para la economia.
(aunque conozco gente que programa como si estuviera fabricando zapatos :-) )
No creo que el software pueda escapar facilmente a la necesidad de
ajustarse a los requermientos particulares de los clientes, con la
exepcion de cosas genericas como sistemas operativos, procesadores de
texto, etc (en gral, lo que apunta al "end-user").
Cada empresa tiene una forma diferente de hacer las cosas, y eso es lo
que le permite competir con otras empresas que hacen lo mismo. Si el
software es uniforme y no se puede adaptar a eso, las empresas no creo
que lo adopten (ej. SAP es muy generico y configurable, pero
implantarlo no es facil, requiere mucho trabajo para adaptarlo a la
empresa).
Seguro, podemos abstraer el dominio(o la arquitectura) usando DSLs o
MDA, pero al final igual se necesita un grupo de artesanos altamente
especializados para interpretar las necesidades y codificarlas.
Estamos logrando avances al subir cada vez mas el nivel de
abstraccion, pero crear software sigue siendo esencialmente dificil (y
"No Silver Bullet" de Frederick Brooks sigue vigente :-) )

Saludos

Gabriel
http://gabrielsw.blogspot.com

Jorge Ubeda dijo...

Gabriel, gracias por reponer aquí lo que conversáramos por correo. Brevemente, repito y amplío lo que ya conversamos.
1. El desarrollo del estilo de MDA no debiera ser rígido. Las posibilidades de definir un modelo a alto nivel permitiría describir un problema con toda la riqueza necesaria. Simplemente, no lo haremos en términos de escritura en un lenguaje (Java, C++, o lo que sea), sino en términos más abstractos,sean visuales o no.
2. Por esta misma razón, un desarrollador trabajando en este terreno afronta una tarea más rica y compleja, aunque mucho más productiva. De ninguna manera el uso de estas herramientas implica despersonalizar el trabajo, sino, por el contrario, exige de los desarrolladores una actitud de mayor visualización sobre su trabajo. No es tan importante la construcción de un algoritmo, como la visión estratégica del modelo en construcción.
Esto es lo que trato de transmitir...

Gabriel C. dijo...

Creo que yo apuntaba justamente al punto 2.
(Hasta donde yo entiendo) los DSL y MDA elevan el nivel de abstraccion, permitiendo concentrarnos en lo que realmente importa, obviando los detalles "mundanos", logrando que el trabajo sea mas productivo. Y si, ademas requiere una vision un poco mas global del problema.
Mi critica al articulo original es que me parece que la "industrializacion" del software si no es una fantasia, es una metafora que se queda corta. Con MDA/DSLs o sin, el software se trata de diseñar un producto, no de fabricarlo (en el peor de los casos, es configurar un producto existente a las necesidades particulares). Una fabrica tiene una economia de escala en el momento de la produccion, pero cuanto tiempo y dinero se gasta en diseñar el producto? Que pasaria si esa fabrica tuviera que ajustar cada producto a las necesidades particulares de cada cliente?. En el corto o mediano plazo, no veo una solucion que evite depender de "pequeños grupos de artesanos altamente especializados" (aunque estos usen MDA y DSLs).
Estoy totalmente convencido de que debemos movernos hacia "la 4ta generacion" con MDA y DSLs (me encantaria tener la oportunidad de probarlos) pero me parecio que el articulo era medio "engañoso" (y no queria desaprovechar la oportunidad de participar ;-) )...

Jorge Ubeda dijo...

Bueno...el artículo referido fue uno de los primeros de la serie sostenida por Microsoft en defensa de su punto de vista sobre Software Factories. Una búsqueda en Google da una idea de esta actividad. El artículo trabaja en la dirección de promover las herramientas de Visual Studio diseñadas dentro de esta orientación:
While all of this conjecture sounds wonderful for the future, many developers will be asking themselves what they can do now to utilize Software Factory techniques in their organizations today. Well, the good news is that a lot of functionality already exists with Visual Studio.NET. Products like Enterprise Templates, Project Item Templates, and Nant allow for the creation of standard artifacts that a team or organization can utilize today..
Siendo la iniciativa de verdadero interés, en varias ocasiones debí sin embargo señalar mi desacuerdo con el otro aspecto de esta iniciativa, no muy visible en esta nota todavía, pero bien expresada luego , que consiste en desacreditar las iniciativas de la OMG.
Considero que el tema no está cerrado ni mucho menos, sino que recién se abre, y queda camino por recorrer, que probablemente tome elementos de una y otra visión.
En cuanto a la "visión fabril", en mi parecer es sumamente interesante, y en realidad es una derivación necesaria de varias líneas confluyentes: el uso de componentes (el mercado de activex existe desde hace años), MDA, DLS, SPL. Software Product Lines en todo caso no puedo asimilarlo a una planta de ensamblaje automotriz, pero sí creo que se puede llevar a lo que es un proyecto de ingeniería, digamos la construcción de una represa y o un puente.