(Short Answers) 1. Explain the difference between shared and distributed memory.

Question

(Short Answers)

1. Explain the difference between shared and distributed memory.

2. Give an example of an ordering of accesses among concurrent activities (e.g., a program with a data race) that is not sequentially consistent.

3.What characteristics make it possible to convert a serial program to a parallel program?

4.Parallel programming involves trading off the complexity of distributing computation and the overhead of communications for the ability to perform multiple computations at once. Describe a case where it may be possible but undesirable to parallelize a serial program.

Explanation / Answer

1) The architectural differences between shared-memory systems and distributed-memory systems have implications on how each is programmed. With a shared-memory multiprocessor, different processors can access the same variables. This makes referencing data stored in memory similar to traditional single-processor programs, but adds the complexity of shared data integrity. A distributed-memory system introduces a different problem: how to distribute a computational task to multiple processors with distinct memory spaces and reassemble the results from each processor into one solution.

3)

conersion can be done two ways
automatic and manual.

Automatic parallelization

There are still too many interlocking factors, too complex to be captured in code, that require even the best automatic tools to be used in conjunction with well-trained conversion experts.

ex:: do -loops

Manual parallelization

    Programmer must spend time to parallelize

    Expect this to be a time-consuming process, and budget for it ...if you expect to simply have to change a few lines, then you've likely got a nasty shock in store.

    Some possible actions ("Rules of thumb")

    Over the years, a body of hard-learned wisdom has grown regarding how one can most efficiently extract parallelism from serial seeds; here are a few:

        Remove inhibitors to parallelization

        Unnecessary serialization, for example, making all processes wait while one of them does something that could have been put off until a later required serial section.

        Re-arranging of loop-indices, to minimize inter-loop dependency.

        Insert constructs or calls to library routines

        Some packages of often-used algorithms, for example, linear algebra routines, have already been parallelized. If your application has a need for such tasks, use someone else's work if at all possible.

        Constructs are commented-out keywords that are intended to be read by pre-processors and code-generators, allowing the programmer to indicate what kinds of parallelism should be attempted in certain parts of the program. Sometimes directives, which insist that things be done a certain way, and sometimes suggestions, indicating potentially useful directions, constructs often make it possible for the programmer to leave a section of serial code completely alone and still have it parallelized because of the actions taken upon recognition of the commented-out information.

        Run code through preprocessors
        Restructure algorithm

Related Questions

Navigate

• Browse (All)
• Browse #
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.