 |
|
 |
Assessment
of student learning: Otto
Rompelman E-mail: o.rompelman@its.tudelft.nl 1. Introduction Everyone involved in engineering education, both students and teachers, know, that in order to get a degree, students have to pass examinations. Examinations are part of the educational culture and this seems to be something like a law of nature. However, our society is changing rapidly and so is the life of the professional engineers we educate. Over the last few decades we have seen a change in the positions engineers take. Engineers, after having obtained their degrees, were supposed to have sufficient academic qualifications to start a life long career. The rapid changes in society have also generated a demand for more flexible engineers having many more qualifications than just a high level of technical or scientific specialisation. These demands have led to an evolution in educational objectives. In the past we tried to transfer knowledge and specialist skills. High quality education was supposed to be guaranteed by the appointment of experienced specialists in the field. Nowadays universities tend to think in terms of much broader skills, one of the main ones being the ability to learn, not only during the time in college but also in the professional life. As a consequence, educational methods are under constant dispute. New forms are introduced, such as teamwork, problem based learning, design education etc. It is therefore not surprising, that assessment is an issue for permanent discussion as well. In this paper we will discuss some aspects of the changes in educational objectives and their consequences for assessment. Two examples from the author’s experience of new educational approaches will be discussed in more detail.
2. The paradigm shift in engineering education Like any education system, engineering education is an issue of continuous discussion. In particular during the last decades there has been a continuous debate on what students should learn, which knowledge and skills are required for good engineering practice, what new educational tools could be introduced for better education, what the role should be of modern technologies such as computer assisted learning or the internet, etc. Educators are overwhelmed by questions and requirements from different parties such as the university board, their fellow teachers, the government, industry, student’s etc. Very often the simplest solution is just keep doing what has been done before. However, it cannot been denied that today’s world is different from the world of, say 30 years ago. Even more, the world of the next century will certainly be different from today’s world. We should realise that right now we are educating young people that will be professionally active in the first decades of this new century. It is therefore essential that we have a clear view on our educational goals and, consequently, the educational system. In this chapter we will discuss the backgrounds of the paradigm shift in engineering education which can be roughly defined as a shift from teaching to learning. A system's approach to education A good engineering education system is aimed at the formation of good engineers. We can describe the education process as a transformation in which incoming students are ‘transformed’ into engineers as is depicted in Figure 1.
Figure 1: Education as a 'process of human transformation'. It is well known from systems theory, that, in order to describe a transform we need to define both the input and the output. In other words, both the characteristics of the incoming students and the characteristics of the young starting engineers should be clearly defined. A further definition of the characteristics of the student is beyond the scope of this paper. Let us now concentrate on ‘the output’. Engineering educators should have a clear view of what a good engineer is, in other words: the goals of engineering education should be clearly defined. Once these goals are defined we can try and develop the engineering education system. As far as the educational objectives are concerned, I think that we should keep in mind that the goals and aims of engineering education have to be defined by the university using the actively acquired vision of the graduates themselves, the future employees of these graduates (industry) and the society as a whole. The relation between education and the ‘outside world’ is depicted in Figure 2.
Figure 2: Influences on the educational structure It is the author’s opinion that, whatever the external world puts forward, it is the final responsibility of the universities how and to what extent these views are to be converted into educational objectives. 1-It
should be noted, however, that prior education an important factor
that everyone involved in undergraduate education should take into
account. Simply taking the view that they know (or should know)
particular things is not sufficient. Complaining that present freshman
have a lack of knowledge doesnąt solve their problem. We have to
either cope with it or try and change the prior education. Obviously,
the first solution is more practical! This doesnąt mean, that, at
the same time, we should not try the latter.
The evolution of educational objectives It will now be discussed how the educational objectives have changed in the last couple of decades. Traditionally, engineers are supposed to have three important intellectual qualities:
Up till the fifties, it was assumed that the education was successful if the graduate had acquired the knowledge and skills that were sufficient for starting a career of, say about 40 years. In his paper Bucciarelli of MIT states [1]: The 50’s type is well prepared in the sciences. He applies this knowledge to complex high tech problems of the kind encountered in the engineering of complex military-industrial and aerospace systems. He does this as a staff employee with a large, well equipped, well organised and authoritarian organization: he toils within a bureau defined by its technical focus and expertise. The curriculum leading to this type of engineer puts a large emphasis on knowledge and specialised skills. As a consequence of the emphasis on knowledge, the educational methods were (and often still are) largely focussed on teaching. The leading question in education was (somewhat overstated): "how do we as teachers get all the knowledge into the heads of the students?" The ‘modern’ engineer, however, is less a specialist but has a number of other abilities. As Bucciarelli in the same paper states [1]: The ideal type of the 90’s is of a different sort. Our 90’s graduate is also well prepared in the fundamentals, but now of a more diverse set of disciplines. Prepared to work in teams, able to articulate, communicate and defend a proposal. The 90’s type is open to negotiations, knows how to cope with uncertainty. Petty in an interesting paper summarises that engineering curricula should produce graduates that have [2]:
This list can be considered as a set of specifications, that graduates should meet. Consequently, the curricula now shift from being mainly focussed on pure knowledge and specialist skills to paying attention to understanding and more general skills. In the paper of Petty this is reflected in the change of curriculum contents of engineering degrees as proposed by the Engineering Professors Conference in the U.K. (Figure 3, [2]).
Figure 3: The change of emphasis in the contents of engineering curricula [2] This shift is not simply effectuated by the replacement of some courses by other ones. A closer look at the aforementioned specifications leads to the conclusion that traditional courses don’t cope with these objectives. How do we teach understanding, design ability or problem solving skills ? It appears that the earlier mentioned three intellectual qualities knowledge, skills and understanding don’t sufficiently cover our newly defined objectives. Recently the term competency has appeared in the discussions. Though there has been a lot of discussions on what is actually meant by competence, we will define it here as the combination (or better: integration) of knowledge, skills, personal characteristics and experience. It is indeed competency that has become a key quality of the graduate. Competencies in professional life are characterised by [3]:
Competencies are under constant development. Competency development is not restricted to the professional life of the engineer. In our educational system we should prepare on this development and try and include elements in the curricula that prepare engineers on competency development by, what is now often referred to as life long learning. In the seventies, it became clear that the evolving educational objectives could no longer be arrived at by traditional educational methods like lectures and laboratories. Furthermore, investigations on what students had actually learnt, showed, that there was a growing discrepancy between the actual knowledge and skills of the students and the expectations of their teachers. In different places experiments were started with new educational approaches such as problem based learning (PBL) and project organised learning (POL). We refer to [4] for further information. Rather rigorous experiments were started in the University of Aalborg in Denmark (to be discussed later) and the medical school of the University of Maastricht in the Netherlands [5]. In general it can be stated that there should be a great emphasis on aspects as integration of knowledge, creativity, design ability and communication, in other words: graduates are supposed to be active. How can we expect our graduates to behave as active and creative people if we trained them for four to five years in silently listening to what their teachers tell them? So, if we indeed want our graduates to meet the specifications, we need to change our educational systems. We have to shift our focus from teaching (teacher oriented programmes) to learning (student-oriented programmes) as depicted in Figure 4.
Figure 4: Development from teacher oriented to learner oriented education. It is important to note that we have to incorporate these changes not because they are fashionable but because they confirm the needs of the customers of our ‘products’: the young engineers.
3. Relation between educational methods and assessment In this chapter we review the function of assessment in education in general. We will discuss how educational objectives are related to educational methods and assessment. What is assessment? In the previous chapter we have defined education as a transformation process. Assessment of student learning is in fact the assessment whether this transformation was successful. This question can only be answered if the educational objectives are explicitly defined. Now, assessment is simply comparing the results obtained by the student with these objectives as depicted as in Figure 5.
Figure 5: Place and function of assessment It is interesting to note that it was the introduction of problem based learning as mentioned in the previous chapter, that it became indeed possible to derive tests directly from the educational objectives. At this time it is important to have a closer look at the educational process. There are two important actors in this process, viz. the teacher and the learner (student). Their roles can be characterised as follows:
In the student centred vision on education the student plays the key role, whereas the teacher’s main function is to create an optimal environment for the student to learn. Sometimes the main role of the teacher is described as the manager of the educational process. Referring to Figure 5, this means that teachers should to do three things:
This can be illustrated with the following example. Let us assume that the educational objective is that the student should know the first law of thermodynamics. The teacher can now try and find ways for students to arrive at the objective such as:
Consequently he has to find a way of assessing whether the student now indeed knows this law. This is a very simple example, since the objective is either met or not. It is related to purely reproductive knowledge. The situation becomes much more complicated, if the objective is not simply knowing the law, but also understanding it. This is at the same time a much more important objective if we look at the set of specifications discussed in the previous chapter, one of the reasons being that once something is indeed understood, it may be used in a much more creative way than just reproducing it. In Chapter 4 this will be discussed in more detail. What kind of teaching method should be used in order to achieve the understanding? Very often teachers suppose that explaining something is sufficient. It is common experience, however, that students are often very capable in reproducing knowledge, but that they fail in understanding it. At this time, it is important to ask the question: what happens if the student fails, or, relating this question to Figure 5: what, if the result is negative? The obvious answer is that the student has to do the test again. A more valid answer, however, is that something in the educational process didn’t work as was envisaged. This implies both the student (as was initially assumed) and the teacher. The conclusion therefore is that we have to find out the reasons for failing and, consequently take measures to improve results the next time. This is depicted in Figure 6.
Figure 6: The role of assessment in the educational process From this figure we can conclude that, if the results are not in agreement with the objectives, action can be taken with respect to I:
the assessment procedure (short term feed back) All possibilities should be investigated and sometimes more than one of the feed back paths should be followed. The conclusion is that assessment plays a role that goes far beyond the traditional assessment whether students meet the requirements. From Figure 6 it becomes immediately clear that, because of the feed back loops, education has now become a dynamic system with all attractive but also negative aspects of this type of systems. The positive effect is that we now appreciate, that we can adapt to changes. Furthermore, on the basis of this system it is possible to include the issue of quality, in other words: assessment can now be used to evaluate the quality of our entire educational system, including the students.
4. Educational methods and their consequences for assessment The shift from rather isolated knowledge elements and skills to a more integrative approach has large implications on the teaching methods and, therefore, on the assessment methods. In this chapter we will discuss how the change in educational objectives leads to a change in assessment culture. Furthermore, two examples of the tight relation between objectives and assessment are discussed in more detail. Assessment: a shift in culture Traditionally, assessment was mainly based on verifying whether educational goals are reached. This means, that it was tested, whether students had acquired the desired knowledge, skills and understanding. This approach, however, is less applicable if we (teachers, students and examiners alike) want to find out whether contemporary objectives, as they were mentioned in the previous chapter, are met. It is not surprising that a change in attitude towards assessment can be observed. This is clearly illustrated in a paper by McDowell, in which she distinguishes two cultures: the testing culture and the assessment culture [6]. She discusses the two cultures by looking at
We will shortly review how the testing and assessment cultures can be characterised with respect to these three issues. As far as the issue what is assessed is concerned, in the testing culture it is the intention to assess the result of the learning process (acquired knowledge). The assessment culture goes a step further in that the process is assessed by which the results were achieved: how was the information handled, how was it analysed, how were solutions created (synthesis), to what extent was this process critically evaluated etc. The differences are summarised in Table I.
Table I: Dimensions of assessment culture: what is assessed [6] In the testing culture the question how learning is assessed is approached by standardised methods that are the same for all students. The obvious example is a written examination, in which problems are given that have to be solved, usually in the form of calculations. In the assessment culture it is acknowledged, that judgement and a critical view on criteria are required, which leads to context dependent assessment procedures. The differences with respect to how learning is assessed are shortly displayed in Table II.
Table II: Dimensions of assessment culture: how learning is assessed [6] Finally, the question what is the purpose of the assessing learning. This has been discussed already in the previous chapter (see also Figure 6). In addition to what was mentioned there, we see that in the testing culture the emphasis is on ranking. This requires an accurate and quantitative measurement of the abilities or potential of the students. The assessment culture is more focussed on comprehensively describing the performance of the students. It is intended to present the range of the achievements in a more qualitative way rather than a quantitative way. Table III gives some the differences of the cultures with respect to the issue of the purpose of assessing learning.
Table
III: Dimensions of assessment culture: the purpose of assessing
learning Assessment of basic knowledge As mentioned before, very often students in their examinations are very well able to reproduce facts and figures. However, if tested for real understanding, the results are much poorer. Rump, Jacobsen and Clemmensen have reported on an interesting research project aiming at a quality development model for education [7]. They found that students often pass their examinations without having sufficiently understanding of the fundamental concepts of the examined subject. First it was analysed what the difficulties were in obtaining understanding of the subject. These difficulties were:
The authors showed that by following the feed back paths I and III in Figure 5 the results could be improved substantially. Feed back path I implied a revision of the questions and problems in the assessment. Feed back path III implied a thorough discussion with the teachers on the concepts, principles, models, techniques and methods in the course contents. Their conclusion was, that though traditional pedagogical training of teachers can improve results, a more thorough analysis of the problems encountered by the students will certainly lead to better results, in other words: the educational objectives are met to a much larger extent.
Assessment of group work Technical problems nowadays are very complex. The impact on society of the solutions has likewise increased. It is not surprising, that indeed, teams rather than single engineers usually deal with contemporary projects. Often these teams are interdisciplinary and in many cases composed of people from different countries and therefore different cultural backgrounds. How do we prepare our graduates for these new and challenging situations, if it is possible at all? The answer to this question nowadays lies in the introduction of group or teamwork in the curricula. In practice this is often implemented in the form of project education, problem based learning, design teams etc. The educational value of working in teams is hardly disputed. The introduction of these methods is often hampered by practical problems. Some of these problems are
For all these questions it has to be clearly recorded, what is to be decided by the teacher and what by the students themselves. Other important factors are communication, definition of the final problem, agreement on methods and presentation of results. The most frequent objection against group work doesn’t arise from the fear of the mentioned problems, however, but from the assessment. In a seminar on assessment of student learning [8] there were short introductions on experience gained with project work in groups, remote group work, i.e. groups of students in different locations, and international student teamwork. Some of the main problems with group work are, what is called, 'hitch hikers'. The problem can be overcome by clear and open comments from the tutor, social pressure through peer assessment and the logging of individual contributions by monitoring the electronic communication. Removing a member from a group doesn’t seem to be appropriate, since the rest of the group is punished with a reduction in capacity. Another problem is was that groups tend to deviate from real 'hard engineering'. It is advised that group projects should be very realistic and, if necessary, the contents should be redefined. Furthermore, a continuous assessment was recommended. Finally, it is suggested that engineering, design and business be integrated in the projects. When focussing on the assessment of group work, a number of interesting points were made. It seems to be appropriate to discriminate between feedback and assessment. Assessment uses an accepted and formalised system and is aimed at assessing whether particular goals have been reached and with what quality. Feedback is used to comment to students in order to achieve better results in the future. Assessment of products was found to be relatively easy, but problems arise when assessing actions within a group. Furthermore, in practice many teachers are involved in a course consisting of group work. This leads to the problem of different standards. It can be concluded that regular assessment of teachers and a good management of the entire course are absolutely necessary. It may be questioned whether peer assessment sufficiently solves this problem. Finally, some negative aspects for students have been reported such as the fact that often other courses run in parallel with projects and the problem that extra activities, required to get a sufficient marking (after the project term), might be in conflict with other obligations. A new range of problems are encountered, when students work in a group without meeting face-to-face, or when students are from different nationalities. Again, the main (and most important) issue is the necessity to make an inventory of these problems when they occur. There are no clear-cut solutions. In the case of remote groups, some of the problems are the high dependency on technology (ICT), the lack of body language and, partly due to that, the lack of immediacy and the slow decision making and the lack of pressure for progress. A meeting of the group members prior to the project is almost inevitable. As far as groups of international composition are concerned, the main problems identified seem to be the obvious language problems (both for students and teacher), the differences in cultural background leading to for instance different expectations as to both learning and teaching style.
5. Practical examples of innovative programmes This chapter doesn’t pretend to show the most striking innovations in engineering curricula. Many technical universities are introducing (or already apply) new educational approaches in their curricula. Sometimes this regards small and carefully introduced experiments. Also, curricula are introduced, that are fully based on project and/or problem based learning. As an example we refer to the Aalborg University in Denmark [9]. Right from its start, about 25 years ago, the curricula were developed such, that they heavily rely on project work. This means that it is through projects, that the necessity of knowledge areas are appreciated and consequently taught in courses. At average about 50% of the study time is spent in projects. The experience gained with this approach has been reported recently [10]. In this chapter two examples from the authors Department will be discussed in more detail. It will be shown that indeed it is possible to incorporate many of the objectives of engineering education as they were mentioned in the previous chapter. Example I: the Integrated Working Groups Electrical Engineering The Electrical Engineering programme is one of the 15 different programmes of the Delft University of Technology (Netherlands). A number of problems were identified in the first year curriculum. Some of them were:
It was decided to introduce a new module (‘course’), which was called Integrative Working-groups Electrical Engineering (IWEE). The relative contribution to the curriculum is about 15% of the entire first year programme (5.5 credit points of the 42 credit points). The educational objectives were defined in term of learning goals:
The educational setting is as follows. All first year students are placed in groups of 10. Each group has a mentor (teaching staff member). The course is co-ordinated by a teaching staff member (associate professor). The group activities comprise (note that the academic year consists of four terms per year):
A detailed discussion on the different activities is beyond the scope of this chapter. Further details are described elsewhere [11]. However, in order to give some idea about the activities of the group we will shortly describe two of the projects. One project is concerned with the analysis of s some simple electrical networks (RLC). The basic idea is that they try and predict the behaviour of the circuit in a rather qualitative way. Consequently, the analysis is carried out in three ways:
Usually, teams of two students do the work. When the work is done, they meet again in the team and report on their findings. The main goals now is:
This latter discussion is used to clarify the concept of models (mathematical, physical, computer), their pre-assumptions and their accuracy. The project is concluded with a written report summarising both the findings and the results of the discussion. The tutor plays a role on the background, but stimulates the final discussion, mainly by asking questions. This project takes one half day (4 hours). Another project is the analysis of a CD player. Here the tutor takes a more active, but still co-ordinating role. By mean of asking questions in an interactive the students more or less design a CD player on the basis of analysing which functions are needed to produce music out of a CD. Functions are: rotating mechanism, a system for tracking, system for reading the information from the disk, system to decode the signals from the disc, a system to create an analogue signal ready for amplification, a system to control the player (remote control) etc. The tutor on the basis of suggestions from of the students draws a rough function block diagram on a white board. When the students think all relevant functions have been identified and have been placed in the functional block diagram, the students are invited to ‘clean up’ the diagram. Now they split up the complete system in e.g. 4 parts and subgroups will study the implementation of the different sections. This study is done as homework. The next week they meet again and the report orally (using overhead projection) on their findings. Questions are asked and a final report is composed. Finally, the tutor invites the students to identify the relation between their first year courses and the implementations they have discussed. This project takes two half days (2*4 hours) plus homework for studying and preparing the presentations. During the year different group members play the role of chairperson and secretary (in charge of written group reports). Though the students often split up in subgroups, all members are supposed to have sufficient knowledge about the results of the group mates: this is checked during the assessment sessions (to be mentioned hereafter). This forces the students to effectively and efficiently communicate and to create clear reports. The assessment is done by the tutors and is twofold:
The tutors are supplied with checklists with a number of items such as active participation, preparation, group behaviour, level of conceptual thinking, quality of oral presentations, quality of written documents. The course has been in operation for four years and the results are very interesting. From questionnaires and discussions with the students it was found that they had learned to
Furthermore, they have discovered they are now able
Their motivation increases over the year. As some students reported: ‘We were able to do more than we ever considered to be possible which is highly stimulating and motivating’ Some observations of the teachers were:
Some general conclusions are:
Example II: the Integrated Design Project One of the important objectives in present and future engineering education is the ability to design. In 1993 the Integrated Design Project was introduced in the Electrical Engineering curriculum of TU Delft. Just as the first year project discussed above, this project was developed in order to solve a number problems identified at the end of the third year (comparable to the BSc-level). There seemed to be a lack of
In order to solve these problems a number of (rather ambitious) educational objectives were defined for the Integrated Design Project:
Some of the objectives are similar to the objectives formulated for the IWEE as discussed above. This is partly due to the fact that the IDP was developed and introduced a few years before the IWEE and partly because we think that a number of issues need repetition in orders to improve quality. The IDP is in fact an exercise in structured design. The underlying philosophy on design is discussed elsewhere [12]. The design methodology is based on the so call seven-stage model of the integrated product life cycle as shown in Figure 7 [13].
Figure 7: The seven-stage model of the integrated product life cycle The students work in teams of two. The follow the process as depicted in Figure 7 in a structured way such that three phases can be identified. In Phase I the take the following actions:
The demands (requirements) are used on 3 levels 1- as a source of criteria to evaluate the different possible solutions 2- as a necessary starting point for the design process 3- as a verification protocol for the final product An important issue is the fact that there are limited resources viz. the abilities of the team, the budget agreed upon with the client and the available time. In Phase II the actual product is made. This product can be hard ware, software or a combination of the two. In Phase III the product is finalised. The different documents are produced. These include a report, instruction manuals for putting in use, for operation, for maintenance and for disposal. The teaching staff involvement consists of four:
At the end of the project all documents are submitted to an examination committee consisting of the four above mentioned staff members. The project is concluded with an oral presentation and, if possible, a demonstration of the product. At the end of their presentation the students have to evaluate their own activities. This implies a critical review of what they have done what went wrong and how problems were solved. The assessment is reasonably well in agreement with the Assessment Culture as discussed in the previous chapter. A checklist is used consisting of three groups of elements headed by the three main questions:
The examination committee, in a discussion on these items, formulates the final assessment. Since the department requires individual marks, the examination committee in a discussion on these items translates the assessment into markings (one mark for each of the three mentioned issues) and a final mark. The result is communicated to the students immediately after the deliberation. Only in clear cases of different contributions of the team members the committee deviates from giving the same mark to both students. After five years of experience with this project it can be concluded that the students have experienced to
The conclusion is justified that the project is a great success. It is envisaged to broaden the scope of the project in the sense students from other field are included (multidisciplinary teams) or that students from universities in different European countries form teams (multinational teams).
6. Concluding remarks The assessment of student learning has a much wider impact than just marking examinations or papers of students. Assessment is highly related to the goals of engineering education. In a continuously evolving society these goals are under permanent discussion. This discussion is reflected in an adaptation of educational methods. Its is therefore natural that also assessment methods should be evolving together with education. A lot of interesting experiments are going on all over the world. Present means of communication will allow engineering educators to keep informed about developments elsewhere. Many engineering education societies such as the ASEE (USA) and SEFI (Europe) are continuously paying attention to the development of education, educational methods and assessment. Only through intensive exchange of experiences it is possible to keep up with the ever-changing demands.
References [1] L. Bucciarelli, Educating the learning practitioner In: F. Maffioli, M. Horvath and F. Reichl (eds.), "Educating the Engineer for Lifelong Learning; Proc. SEFI Annual Conf., Vienna, 1996", pp. 13-22 [2] I. Petty, Vision 2020 — Education in the next Millennium In: A. Hagström (Ed.): "Engineering Education: Rediscovering the Centre; Proc. SEFI Annual Conf., Winterthur and Zürich, 1999", pp. 27-35 [3] C. Levy — Leboyer, L’ambition professionelle et la Mobilité Sociale, Presses Univ. de France, 1971 [4] D.A. Kolb, Experiential learning. Experience as the source of learning and development, Prentice Hall, New Yersey, 1984 [5] see: http://www.unimaas.nl/~PBL/ [6] L. McDowell, Assessment and learning: some insights from current research In: J. Daudt and O. Rompelman (eds.): "What have they learned? Assessment of Student Learning", SEFI Document No.23, SEFI, Brussels, 1999, pp. 7-13 [7] C. Rump, A. Jakobsen and T. Clemmensen, Improving conceptual understanding using qualitative tests In: J. Daudt and O. Rompelman (eds.): "What have they learned? Assessment of Student Learning", SEFI Document No.23, SEFI, Brussels, 1999, pp. 15-26 [8] J. Daudt and O. Rompelman (eds.): "What have they learned? Assessment of Student Learning", SEFI Document No.23, SEFI, Brussels, 1999 [9] F. Kjersdam and S. Enemark, The Aalborg Experiment — project innovation in university education, Aalborg University Press, 1994 [10] F.K. Fink, Integration of Engineering Practice into Curriculum — 25 years of experience with Problem Based Learning, Proceedings of IEEE conference ‘Frontiers in Education (FIE’99), Puerto Rico, November 1999 (to be accessed via: http://fairway.ecn.purdue.edu/~fie/fie99/) [11] O. Rompelman and M.J. Vellekoop, A first year course in integrative learning: a practical example of ‘back to the basics In: A. Hagström (Ed.): "Engineering Education: Rediscovering the Centre; Proc. SEFI Annual Conf., Winterthur and Zürich, 1999", pp 225-230 (to be accessed via: http://bosz.its.tudelft.nl/rompelman/paper-IWEE.htm [12] O. Rompelman and W. ten Haaf, What is design? A framework for design methodology In: V. John & K. Lassithiotakis (Eds.): "Design in Engineering Education; Proceedings of the SEFI-CDWG Seminar, Odense, Denmark, 22-24 October 1998", pp. 97-101 (to be accessed via: http://bosz.its.tudelft.nl/rompelman/paper-design.htm) [13] O. Rompelman, The integrated design project in the Delft Electrical Engineering curriculum In: F. Maffioli, M. Horvath and F. Reichl (eds.), "Educating the Engineer for Lifelong Learning; Proc. SEFI Annual Conf., Vienna, 1996", pp. 333-338 (to be accessed via: http://bosz.its.tudelft.nl/rompelman/paper-IDP.htm)
|
|||||||||||||||||||||||||||||||||||||||
