CIA-RDP96-00788R001800050001-8

10
SRI Internationa
SECRET
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Final Report
December 1982
REVISED JULY 1983
A REMOTE VIEWING EVALUATION PROTOCOL (U)
By:
EDWIN C. MAY
Prepared for:
DEFENSE INTELLIGENCE AGENCY
WASHINGTON, D.C. 20301
Attention:
DT-5A
SG1J
CONTRACT MDA908-82-C-0034
SPECIAL ACCESS PROGRAM FOR GRILL FLAME.
RESTRICT DISSEMINATION TO ONLY INDIVIDUALS WITH VERIFIED ACCESS.
C
CLASSIFIED BY: DT-5A
REVIEW ON: 31 December 2002
Copy No.
1
This document consists of 48 pages.
SRI/GF-0247
SRI
International
333 Ravenswood Ave. . Menlo Park, California 94025
NOT RELEASABLE TO
(415) 326-6200 . Calad -ST
· TWX: 910-373-1246
Approved For Release 2000/08/08 . CIA-RDP96-00788R001800058EN NATIONALS

Approved For Release 200bN:CLASSIFIED 0788R001800050001-8
CONTENTS (U)
LIST OF ILLUSTRATIONS (U).
iii
LIST OF TABLES (U)
iv
I
OBJECTIVE (U)
1
II
INTRODUCTION (U)
.
2
-
III
ANALYSIS PROTOCOL (U)
4
IV
CONCLUSIONS (U)
11
Appendix A -- EVALUATION OF FOUR COORDINATE REMOTE VIEWINGS (U). .
12
Appendix B -- SUMMARY OF EVALUATION TECHNIQUE (U).
43
REFERENCES (U.
44
-
-
-
-
ii
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

Approved For Release 20000066! ASSI5PE:D0788R001800050001-8
-
FIGURES (U)
1
Sample RV Assessment Form (U).
5
A-1
Transcript 1 (U)
14
A-2
Transcript 2 (U)
15
-
A-3
Transcript 3 (U)
16
A-4
Transcript 4 (U)
17
A-5
Target Site Hango (U).
19
A-6
Target Site Fernando de Noronha (U)
20
A-7
Target Site Inverness (U).
21
A-8
Target Site Punkaharju (U)
22
-
-
iii
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

Approved For Release 2009/dasLASSUFLE D. 788R001800050001-8
-
-
-
1
-
-
-
=
-
-
1
TABLES (U)
1
Task-Defined Relevance Scale for Target Elements (U)
6
2
Quality Assessment Scale (U)
8
3
Numerical Score Conversion Table (U)
8
A-1
Target Element Relevance Ratings for Hango (U)
23
A-2
Target Element Relevance Ratings for Inverness (U)
23
A-3
Target Element Relevance Ratings for Fernando de
Noronha (U).
24
A-4
Target Element Relevance Ratings for Punkaharju (U).
24
A-5
RV Assessment Forms for Calibration of Remote Viewing (U). .
25
A-6
A Rank Ordering of Weighted Averages (U)
41
iv
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

-
1
-
-
-
-
Approved For Release 2008//Sd ASSIF-ED0788R001800050001-8
I
OBJECTIVE (U)
-
(U) The objective of this task was to develop an evaluation procedure
to assess the relative quality of a set of different remote viewing (RV)
responses.
1
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

Approved For Release 2dob.CL A.S.Sk51-5D.0788R001800050001-8
III ANALYSIS PROTOCOL (U)
(U) To quantify the analysis procedure, we have divided the task
into four separate areas: subject response, target/task definition,
quality assessment, and numerical analysis. Figure 1 is a sample RV
Assessment Form that has been designed to emphasize the separation of the
analysis tasks. Each of the parts of the form are described below.
-
-
-
-
-
-
1
-
-
1
-
-
(U) For this initial evaluation technique, a particular concept
should be used only once in the analysis. (Some weighting factor propor-
tional to concept frequency could be utilized, but, for the initial attempt,
only unique concepts are used.) If in the construction of the transcript
concept list a concept later in the transcript is a duplicate of an earlier
one, it should be so noted by placing the concept number of the original
concept in the "D" column.
4
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
(U) The subject's response should be prepared for analysis without
any knowledge of either the target site or the overall task. The aim of
this method of response preparation is to reduce a possibly redundant,
rambling response to a coherent set of concepts. To meet this requirement
we have developed a set of initial guidelines to the conceptualization
procedure.
(U) A concept is defined as a paraphrase of a single idea that has
been expressed in the RV verbal or drawing response. That coherent idea
should not be fragmented into component parts. For example, a response
might be of the form, "I see a large, textured, gray building." The single
concept that expresses this idea should be "large, textured, gray building,"
rather than four separate concepts -- one for each word in the phrase. Each
concept should be entered under the "Transcript Concept" column in the RV
Assessment Form.

1
-
1
1
1
L
-
1
1
I
L
L
L
-
Begin Time:
Target :
Viewer:
End Time:
Method of Targeting :
Date:
Length of Session:
Session:
Class:
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(Q)
PXRXQ
Score
Normalized
Score
1
0
0
0.00
2
2
2
0.71
4
3
3
3
1.07
5
4
4
1.43
6
5
5
1.79
7
8
7
2.50
9
9
8
2.86
10
10
9
3.21
11
12
10
3.57
12
15
11
3.93
13
16
12
4.29
14
20
13
4.64
15
25
14
5.00
Score
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# of Concepts
Approved For Release 20000CLASSIFIED0788R001800050001-8
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
TRANSCRIPT EVALUATION
6
6
2.14
5
8
Weighted Average
1
1
0.36
3
Score
FIGURE 1
SAMPLE RV ASSESSMENT FORM (U)

Approved For Release 2009/00/68 015SUPE 00788R001800050001-8
(U)
To utilize an analysis procedure that is capable of quantitative
assessment, it is necessary to define, in advance, what the goals of the
assessment are. In the RV Assessment Form, columns "Element of Target"
and "Relevance" are provided to clearly define the goal of the analysis.
In the ideal situation, an RV target should be completely specified in
advance. A target typically consists of a number of target elements,
each of which may have varying relevance with regard to the overall RV
task. For any given target, an independent list of target elements should
be prepared. The selection of what constitutes a target element is left
completely to the discretion of the task coordinator. The target element
must be selected with little regard to task relevance (target element
relevance is accounted for later). Because an RV target consists, in
principle, of an essentially infinite number of possible elements, discre-
tion needs to be exercised in the selection process.
(U) For each target element identified for the site, the task
coordinator must define a relevance rating. This rating allows the
coordinator to tailor the analysis to the task requirements.
Table 1
shows the scale that is used for the target element relevance rating.
Table 1
TASK-DEFINED RELEVANCE SCALE FOR TARGET ELEMENTS (U)
(U)
-
Rating
Relevance Scale
1
A target element of trivial interest
2
A target element of minor interest
3
A target element of intermediate interest
4
A target element of major interest
5
A target element of key interest
6
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
-

Approved For Release 2009/da CLASSIFIED 788R001800050001-8
-
(U) For each transcript concept on the RV Assessment Form (Figure 1),
the analyst should attempt to find the element on the list of target
elements that he/she considers to be the best match. The analyst should
be quite liberal in the concept/element matching (i.e., the quality of
the match should be considered at this point in the analysis). If
he/she is able to identify a target element that might be considered a
match to the given concept, a 1 is placed in the "p" (present) column
on the assessment form. If no element can be identified, a 0 is placed
in the "p" column. After making a target element identification, the
selected target element, and its corresponding overall relevance rating
should be entered in the appropriate columns on the assessment form.
(U)
Having identified a corresponding target element for each con-
cept, it is now appropriate to assess the quality of the match. The
quality assessment is done on the basis of how well the single concept
in question matches the selected target element. The judgement is to be
made without regard to any other issues, such as importance of the concept
to the transcript, or importance (relevance) of the target element to the
target. Table 2 shows the quality assessment scale that is used for this
part of the analysis. The appropriate quality score from Table 2 is
entered in the "Quality" column on the RV Assessment Form for each concept
for which a matching target element has been identified.
(U) An intermediate numerical score is computed for each concept
from the relevance and quality (Tables 1 and 2) evaluation as follows:
S' = P X RX Q ,
where P is the value in the "p" column (0 or 1); R is the relevance
evaluation; and 0 is the quality assessment. S' can assume values ranging
between 0 and 25. Table 3 demonstrates how to determine the final score,
S, for a given value of S' for each concept. The conversion table is
7
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-

Approved For Release 2000 NEL AIS-SIFIED0788R001800050001-8
II
INTRODUCTION (U)
(U)
In addressing the remote viewing (RV) evaluation question, we
consider two separate requirements:
· Absolute evaluation of a single response for a
single unknown target.
· Relative evaluation of a set of responses for
a series of known targets.
The first of these is of the most interest in an operational setting.
As
part of the RV enhancement task, we have considered this problem in two
ways. First, by conducting an operational RV session between two calibra-
tion RV sessions, a tentative a priori assessment of operational efficacy
can be determined.
The evaluation is made on the basis of performance
during the calibration sessions, and on the basis of adherence to a pre-
determined session structure.
.
(U) A second technique for an a priori evaluation was explored as
part of the Fiscal Year 1982 program in an audio-linguistic task. This
task provided indications that careful linguistic analysis, when coupled
with technical audio analysis, could yield an assessment in the absence
of knowledge about the target.
(U)
Various techniques have been used in the past
*
in an attempt to
solve the relative evaluation problem. The most common of these was the
simple rank ordering of all responses, as assessed against all possible
targets used in an experimental series. In this procedure, a judge is
presented with n RV transcripts and n target sites. His task is to arrange
*
References are listed at the end of this report.
(U)
2
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
-
-
=

Approved For Release 2000/08/08: GTA RDP96-00788R001800050001-8
=
(U)
-
-
-
1
-
-
1
1
-
1
-
-
=
1
-
-
-
-
the transcripts in order of the best to least match for each of the n
targets. A simple numerical counting procedure is then used to estimate
the likelihood that the judge's transcript/target matches are by chance
alone.
2
This early technique contained little systematic structure for
determining the final order of matches.
(S) The first step toward systematizing the rank order judging
procedure was to preprocess the raw data in the transcript by "concep-
tualizing" both the verbal and the pictorial responses. Conceptualizing
a transcript requires an analyst to paraphrase the transcript into a list
of coherent statements. This concept list is then compared and scored
concept-by-concept to each of the targets in the experiment. The resulting
scores are averaged for each response, and all responses are rank-ordered
on the basis of these scores."
This improved analysis procedure was applied
to a number of experiments within the Technology Transfer Task for INSCOM
in FY 1980.
(U) The problem with the above technique is that there are no guide-
lines as to how the analyst should paraphrase the transcript; furthermore,
the method in which the concepts are to be assessed against the targets
remains undefined. The purpose of the Evaluation Task in FY 1982 was to
identify a procedure that corrected these deficiencies.
3
SECRET
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

-
Approved For Release 2006DBGLASSIF-ED0788R001800050001-8
Table 2
(U) QUALITY ASSESSMENT SCALE (U)
-
Rating
Discrimination Scale
1
2
Poor description; only one or two aspects of the material match.
Fair description; a few aspects of the material match, but a
large ambiguity exists.
3
Reasonable description; many aspects of the material match, but
there remains some ambiguity.
4
Good description; a large number of aspects of the material
matches, but it is possible to conceive of material that
would be a better match.
5
Excellent description; all or nearly all aspects of the
material match.
Table 3
(U)
NUMERICAL SCORE CONVERSION TABLE (U)
S = P x Rx Q
Score
Normalized Score
-
0
0
0.00
1
12
1
0.35
2
2
0.71
3
3
1.07
4
4
1.43
5
5
1.79
6
6
2.14
8
7
2.50
9
8
2.86
10
9
3.21
12
10
3.57
15
11
3.93
16
12
4.29
20
13
4.64
25
14
5.00
8
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
-
-

Approved For Release 20UNCLASSIFIED0788R001800050001-8
(U)
used to eliminate the nonuniformly-distributed gaps in scoring numbers
that occur if one simply uses the product S'. Thus, the final score for
each concept ranges from 0 to a maximum of 14. This conversion table is
provided as part of the RV Assessment Form.
(U) If an assessment of an individual concept is required, the final
score for each concept/target-element match can be related to the quality
assessment scale by using the conversions shown in the third column of
Table 3 and on the assessment sheet. It should be noted, however, that the
integer scores are used to simplify the remaining calculations.
-
(U) To determine a final evaluation of the complete transcript
assessed against a given target, a weighted average of concept scores
is computed. To assist in the calculation of the weighted average, a
tally box score is provided at the bottom of the RV Assessment Form. For
each of the possible scores, 0 through 14, the number of concepts that
attained that particular score are counted. For example, if 3 concepts
were evaluated with a score of 12, a 3 is entered in the box below the
12 score. If the frequency of occurrence of score S, is f., then the final
j
weighted average is computed by
×
·
Ak = 0.357 A
(U) The normalized, weighted average score, A, is entered in the
weighted average box on the assessment sheet. The weighted average score
has been normalized to be within the range
0 _ A _ 5.0
.
To aid in the interpretation of the result, the quality assessment scale
(Table 2) can be used to assess quality of the match between the whole RV
response and the given target site.
(U) At this point in the evaluation protocol, the following options
are available, depending on the task requirement:
9
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
-
-

-
-
-
=
-
-
-
-
Approved For Release 20000EL:A9SIFIED00788R001800050001-8
(U)
. An n X n rank ordering on the basis of the weighted
averages .
. A simple selection of the best match.
. A statistical evaluation on a concept-by-concept
basis.
10
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

SECKEL
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
1
=
1
-
1
I
-
-
-
=
-
-
-
-
-
-
IV CONCLUSIONS (U)
(S) A protocol has been developed to address the relative evaluation
portion of the overall RV transcript assessment problem. As a demonstra-
tion of the technique, we provide in Appendix A an analysis of a series
of four remote viewings that were performed as calibrations during
operational sessions at DIA in December 1981. In this series the remote
viewing products were of relatively high quality, but nonetheless require
a sensitive technique to differentiate because of the similarity of the
targets and, hence, of the descriptions. (The series was chosen primarily
for that reason. ) Application of the assessment technique resulted in
the correct blind matching (highest scoring in matches versus cross
matches) of three of the four.
(U) Appendix B is a one page, step-by-step procedure for the
application of this evaluation technique.
(U) The material in this document thus constitutes an instruction
manual or protocol for application of a step-by-step procedure for quan-
titative assessment of the relative target/transcript correlations of a
series of transcripts matched into a series of targets.
11
SECRET
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

Approved For Release 20gqdag ASS PIED 0788R001800050001-8
1
-
-
-
-
=
-
-
-
Appendix A
EVALUATION OF FOUR COORDINATE REMOTE VIEWINGS (U)
12
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

SECRE
Approved For Release 2000/60/08 : CIA-RDP96-00788R001800050001-8
(S)
On 14 December 1981, four coordinate remote viewings were con-
ducted as calibrations during an operational remote viewing session tar-
geted on client-designated sites of interest. The calibrations and the
operational RV session were conducted at the client's organization and
were witnessed by client representatives. These four calibrations were
chosen as a test bed for the evaluation procedure for the following
reasons, (1) they were conducted in an operational setting, and (2) the
targets had many similar features, and would thus provide a sensitive
test of the protocol.
(S) Figures A-1 through A-4 are the transcripts that were presented
to the analyst. They are exactly as they were when collected, except that
the coordinates have been removed. Figures A-5 through A-8 are the
National Geographic magazine targets that were used during the calibration
sessions. Finally, the task coordinator provided Tables A-1 through A-4
as target element relevance scales for the four targets in Figures A-5
through A-8. This completes the information that was given to the analyst,
and thus the analysis was carried out blind as to the matching target/
transcript pairs.
-
(U)
Table A-5 is a compilation of the completed work sheets that
were used by the analyst in this evaluation. They are shown in groups
by session number, and alphabetized on the four targets. (The task
coordinator first randomized the transcript order then assigned the session
number used above.) For each of the transcripts, the analyst simply
included all phrases and all drawings as concepts. For example, seven
concepts were found during Session 2.
(U)
All concepts were then analyzed as described in the text. The
matching target element, its relevance rating, and the computed score are
shown for all possible combinations of transcript/target pairs in Table
A-5. The score distributions and their resulting weighted averages are
also shown in Table A-5.
13
SECRET
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
-
-
-

Approved For Release 20UNCLASSIFIED0788R001800050001-8
-
-
=
M
TM
Break.
14Dec81
002
B
12.40
-
-
-
A upvcom.
B-
52 rocky
and coastal city
Break
L/W
3
4
Belf.
fjords ..
1
Al Very Beautiful
FIGURE A-1
TRANSCRIPT 1 (U)
14
UNCLASSIFIED
-
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
=
-
-
-
1
-
-
-
-

Approved For Release 2&dUNCLASSIFIED00788R001800050001-8
14 Dec 81
002
3
3.23
-
-
Adl London
n777
Break
1
A straight angles
B -
-
Manoch
2
.
-
Buildings:
End
FIGURE A-2
TRANSCRIPT 2 (U)
15
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
-
-
-

Approved For Release 2000/06: ASS 5-5-00788R001890050001-8
002
1.13
CLASS B
LLI
1
Aridges
Small up /downs
B -
could
-
A up
B
rocky
.
Arising up
B
4
cloudy
snowing
frozen.
clefor with ...
Storming
End
FIGURE A-3
TRANSCRIPT 3 (U)
16
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
TOWN
- feeling of tom.
-
-
-
-
-
-

Approved For Release 200beCLASSIFIED00788R001800050001-8
-
10/Dec 81
002
B
12.46
12.017
1
A down jagged
B -
A Hat
B
water ?
Break
-
-
5.2 green
trees and?
Breads.
-
2
-
-
aoks
munching river.
Jungle.
Break
A clown/up
B deep rally
FIGURE A-4(a)
TRANSCRIPT 4 (U)
-
17
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
-
-

Approved For Release 20bbN: 01 A.S.SI FIED 0788R001800050001-8
-
-
3
A clown
- B-
aol libre mbuside
.
1
desendund
Lw interface
End
FIGURE A-4(b)
TRANSCRIPT 4 (concluded) (U)
-
=
18
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
2
-
-
-

-
-
-
=
-
-
Approved For Release 2
CPYRGHT
20PAPEL8ASISTPRE -0788R001800050001-8
TARGET SITE HANGO (U)
FIGURE A-5
-
-195
UNCLASSIFIED
19
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

-
-
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
CPYRGHT
-
-
-
-
-
-
TARGET SITE FERNANDO de NORONHA (U)
FIGURE A-6
UNCLASSIFIED
20
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

-
-
-
=
=
1
-
-
Approved For Release 3000rotos SGIFTEES-00788R001800050001-8
CPYRGHT
TARGET SITE INVERNESS (U)
FIGURE A-7
UNCLASSIFIED
21
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
CPYRGHT
UNCLASSIFIED
FIGURE A.8 TARGET SITE PUNKAHARJU (U)
22
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

-
1
I
1
1
1
I
1
-
-
1
-
-
-
1
Table A-1
(U) TARGET ELEMENT RELEVANCE RATINGS FOR HANGÖ (U)
Target Element
Relevance
Town
Cold
Peninsula
Rocky
Vegetation
2
Bay
3
5453
5
4
5
3
Table A-2
(U) TARGET ELEMENT RELEVANCE RATINGS FOR INVERNESS (U)
Target Element
Relevance
City
5
River
43
4
3
Bridge
River banks
2
Vegetation
21
1
23
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
UNCLASSLEJER788R001800050001-8
Approved For Release 2000/08/08 .

Approved For Release 200UNCLASSIFIED788R001800050001-8
Table A-3
(U)
TARGET ELEMENT RELEVANCE RATINGS
FOR FERNANDO DE NORONHA (U)
Target Element
Relevance
Island
5
Surf
3
Hills
4
Uninhabited
3
Mountain peak
3
Temperate climate
Vegetation
Ocean
2
224
2
4
Table A-4
(U)
TARGET ELEMENT RELEVANCE RATINGS
FOR PUNKAHARJU (U)
Target Element
Relevance
Connect lakes
5
Town
4
Bridges
2
Cold
4
Vegetation
1
Islands
4
24
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-
-
1
1
£
-
-
-
-
=
-
-
1
-
-
-

1
-
-
-
-
L
1
I
-
-
L
L
L
-
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
12:40
Target:
Inverness
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
1
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
VADCe
(2)
Quality
(9)
Score
PxRxQ
Score
1
Picture 1
1
Twin church towers
1
1
1
0
0
2
Picture 2
1
1
1
3
Up and down
1
Buildings
5
1
5
2
2
4
Rocky
0
3
3
5
Land/water interface
1
River
4
4
12
4
6
Picture 3
1
Twin church towers
1
1
1
5
5
Picture 4
1
River
4
4
12
6
6
8
Uprising
1
Buildings
5
1
5
9
Cliff
0
0
10
Fjords
1
River
5
1
5
10
9
11
Coastal city
1
City
5
4
20
12
10
12
16
12
.
20
13
15
25
14
Score
0
1 2 3
4
567 89
10
11
12
13
14
# of Concepts
2 | 5
33
2
1
TABLE A-5
RV ASSESSMENT FORMS FOR CALIBRATION OF REMOTE VIEWING (U)
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
UNCLASSIFIED
25
8
7
9
8
15
11
13
14
Weighted Average
1.53
TRANSCRIPT EVALUATION

1
1
-
-
1
L
£
1
1
1
L
=
L
L
1
L
L
-
Begin Time:
12:40
Target :
Hangö
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(0)
Score
Score
1
Picture 1
1
Rocks
3
3
8
0
0
2
Picture 2
1
1
1
Up and down
1
Buildings
5
1
5
4
Rocky
1
Rocks
3
5
11
Land/water interfaces
1
Coast
5
4
13
4
6
Picture 3
1
Church
2
1
2
5
5
26
Uprising
1
Slope of land
3
1
3
9
Cliff
1
Coast
5
2
9
10
Fjords
1
Bay
3
11
Coastal city
1
City
5
5
5
14
12
10
12
15
11
13
16
12
14
20
13
15
25
14
Score
0 1
2
2 3 4 5 6 7 8 9
10
11
12
13
14
# of Concepts
421121
1 1
1
1
1
Table A-5 (continued)
Approved For Release 2000/08/08
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
TRANSCRIPT EVALUATION
Picture 4
1
Bay
2
6
6
6
8
7
8
9
8
10
9
1
1
UNCLASSIFIED0788R001800050001-8
Weighted Average
2.52
PxRXQ
2
2
3

Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
12:40
Target :
Fernando de Noronha
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
1
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(Q)
Score
PxRxQ
Score
1
Picture 1
Hills
4
4
12
0
0
2
Picture 2
1
1
3
Up and down
1
Peak
3
1
3
2
2
4
Rocky
1
Coast line
5
5
5
Land/water interface
1
Island
5
4
13
6
Picture 3
1
Peak
3
2
6
5
5
7
Picture 4
1
Island
5
1
5
8
Uprising
1
Hills
4
2
7
9
Cliff
1
Peak
5
3
11
10
Fjords
1
Inlets
3
2
6
11
Coastal city
0
0
12
10
12
15
11
13
16
12
14
20
13
15
25
14
Score
0
1
1
2
3 4
5 6 7 8
9
10
11
12
13
14
# of Concepts
1
2 3 131| 2|1
1 1 1
Weighted Average
2.36
Table A-5 (continued)
Approved For Release 2000/08/08 : CTA-RDP96-00788R001800050001-8
UNCLASSIFIED
UNCLASSIFIED
27
£
1
1
I
-
-
1
.
3
3
4
4
6
8
7
9
8
10
9
TRANSCRIPT EVALUATION
-

Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
12:40
Target :
Punkaharju
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
1
1
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(Q)
Score
PxRxQ Score
1
1
Picture 1
0
0
0
2
Picture 2
1
0
1
1
3
Up and down
1
Town
4
1
4
2
2
4
Rocky
0
0
3
3
5
Land/water interface
1
Connected lakes
5
4
13
4
4
6
Picture 3
0
0
5
5
7
Picture 4
1
Connected lakes
5
2
9
8
Uprising
0
8
7
9
Cliff
0
0
9
8
10
Fjords
1
Connected lakes
5
1
5
10
9
11
Coastal city
1
Town
4
3
10
12
10
12
15
11
13
20
13
15
Score
0 1
2 3
5
6 7 8
1 9
10
11
12
13
14
# of Concepts
521111
1 1
1
Weighted Average
0.90
A-RDP96-00788R001800050001-8
UNCLASSIFIED
Table A-5 (continued)
Approved For Release 2000/08/08:
UNCLASSIFIED
TRANSCRIPT EVALUATION
6
6
28
I
1
1
1
1
-
1
-
L
-
-
16
12
14
25
14
0
0

1
-
-
t
I
L
1
1
1
1
-
I
L
-
-
-
Begin Time:
15:23
Target :
Inverness
Viewer:
#002
End Time:
Method of Targeting:
Coordinate
Date:
14 December 1981
Length of Session:
Session:
2
2
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(0)
Score
PxRxQ
Score
1
Picture 1
1
River bank
4
· 3
10
2
2
Straight angles
1
Buildings
5
1
5
3
Picture 2
1
River
4
3
10
4
River
1
River
5
5
14
5
Buildings
1
Buildings
5
14
Man-made
1
City
5
4
13
London
1
City
4
13
29
7
10
12
10
11
15
11
12
16
12
13
20
13
15
25
14
Score
0
1 2
3
4
5
6
7
8
9
10
11
12
13
14
# of Concepts
1
2
2 2
2
2
2
Table A-5 (continued)
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
3
3
TRANSCRIPT EVALUATION
4
4
5
5
6
6
8
10
9
14
Weighted Average
4.13
0
0
1
1
2
2

Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
15:23
Target :
Hangö
Viewer:
#002
End Time:
Method of Targeting:
Coordinate
Date:
14 December 1981
Length of Session:
Session:
2
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(Q)
Score
Score
1
Picture 1
1
Buildings
5
1
5
0
0
2
Straight angles
1
Buildings
5
1
5
1
1
3
Picture 2
1
Peninsula
5
1
5
4
River
1
Bay
3
2
6
5
Buildings
1
Buildings
5
5
14
4
4
6
Man-made
1
Buildings
5
4
13
7
London
1
Town
5
3
11
30
8
8
7
9
9
8
10
10
9
11
12
10
12
15
11
13
16
12
14
20
13
15
25
14
Score
0
2
3
4
5
6
9
10
11
12
13
14
# of Concepts
3 1 1 1 1
1
1
1
Table A-5 (continued)
Approved For Release 2000/56JASSFJED0788R001800050001-8
UNCLASSIFIED
1
1
1
1
-
I
1
-
1
2
2
3
3
TRANSCRIPT EVALUATION
5
5
6
6
Weighted Average
2.72
PXRXQ

L
I
-
1
I
I
I
1
L
L
I
L
1
Begin Time:
15:23
Target:
Fernando de Noronha
Viewer:
#002
End Time:
Method of Targeting:
Coordinate
Date:
14 December 1981
Length of Session:
Session:
2
Class:
B
Concept
Number
Transcript Concept
D
Element of Target
Rele-
vance
(R)
Quality
(0)
Score
PxRxQ| Score
1
Picture 1
0
0
0
0
2
Straight angles
0
0
1
1
3
Picture 2
1
Island
5
1
5
4
River
1
Ocean
3
1
3
5
Buildings
0
0
6
Man-made
0
0
7
London
0
0
31
8
7
8
9
8
9
10
9
10
12
10
11
15
11
12
16
12
13
20
13
14
25
14
Score
0 1
2
3
4
5 6 7 8
9
10
11
12
13
14
# of Concepts
5
2
1
1
Table A-5 (continued)
Approved For Release 2009/J&CLASSIFIED 788R001800050001-8
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
2
2
3
3
4
4
TRANSCRIPT EVALUATION
5
6
6
15
-
Weighted Average
0.22

I
-
I
-
L
I
-
-
-
L
L
-
-
Begin Time:
15:23
Target :
Punkahar ju
Viewer:
#002
End Time:
Method of Targeting:
Coordinate
Date:
14 December 1981
Length of Session:
Session:
2
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(0)
Score
PxRXQ
Score
1
Picture 1
1
Bridge
2
1
2
0
0
2
Straight angles
1
Bridge
2
1
2
3
Picture 2
1
Bridge
2
1
2
4
River
1
Lakes
5
2
9
5
Buildings
1
Town
4
5
13
6
Man-made
1
Town
4
4
12
7
London
1
Town
4
3
10
32
8
8
7
9
9
8
10
10
9
11
12
10
12
15
11
13
16
12
14
20
13
15
25
14
Score
0 1 2 3
4
5 6 7 8 9
10
11
12
13
14
# of Concepts
3 |1 1 1 1
1
1
£
1 | 1
Table A-5 (continued)
Approved For Release 2000INUCLASSIFIED.788R001800050001-8
UNCLASSIFIED
3
3
4
4
5
5
6
6
TRANSCRIPT EVALUATION
1
1
2
2
Weighted Average
2.11
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

1
I
1
I
L
I
-
-
I
I
I
L
L
L
Begin Time:
13:13
Target :
Inverness
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
3
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(0)
Score
PxRxQ Score
1
Picture 1
1
Buildings
5
1
5
0
0
2
Land/water interface
1
River
4
4
12
1
1
3
Ridges
0
0
4
Small ups and downs
1
Buildings
5
1
5
5
Cold
1
Location
3
1
3
6
Picture 2
1
Church
2
1
2
7
Rocky
0
0
33
8
Picture 3
1
Buildings
5
1
5
0
0
9
8
10
Frozen
0
0
11
Feeling of town
1
City
5
4
13
12
Cliff on water
1
River bank
1
1
1
15
11
13
16
12
14
20
13
15
Score
0
1
2 |3
4
5
6 7 8 9
10
11
12
13
14
# of Concepts
4 6 1 1 2
3
1
1
Table A-5 (continued)
Approved For Release 2005UNCLASSIFIED788R001800050001-8
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
TRANSCRIPT EVALUATION
5
5
6
6
7
Picture 4
10
9
12
10
25
14
Weighted Average
1.12
2
2
3
4

1
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
13:13
Target :
Hango
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
3
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(0)
Score
PxRXQ
Score
1
Picture 1
1
Rocks
3
2
6
0
0
2
Land/water interface
1
Peninsula
5
4
13
1
3
Ridges
1
Rocks
3
2
6
2
2
4
Small ups and downs
1
Rocks
3
2
6
3
3
5
Cold
1
Location
4
4
12
TRANSCRIPT EVALUATION
6
Picture 2
1
Church
2
1
2
7
Rocky
1
Rocks
3
5
11
8
Picture 3
1
Sloping rocks
3
2
6
9
Picture 4
0
0
9
8
10
Frozen
1
Location
4
4
12
10
9
11
Feeling of town
1
Town
5
5
14
12
10
12
Cliff on water
1
Sloping rocks
3
2
6
15
11
13
20
13
15
25
14
Score
0 1 2
3
14
5
6
7 8 9
10
11
12
13
14
# of Concepts
1 15
3 |2 |5
1
2 1
1
Table A-b ( continued)
Approved For Release 2000UNCLASSIFIED 788R001800050001-8
UNCLASSIFIED
1
L
L
I
-
-
-
34
8
7
16
12
14
Weighted Average
2.65
5
4
5
5
6
6
1
1

Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
13:13
Target :
Fernando de Noronha
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
3
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(Q)
Score
Score
1
Picture 1
1
Hills
4
2
7
2
Land/water interface
1
Island
5
4
13
1
1
1
1
3
Ridges
1
Hills
4
5
13
2
4
Small ups and downs
1
Hills
4
3
12
3
3
5
Cold
0
0
6
Picture 2
1
Peak
3
3
8
5
7
Rocky
1
Shoreline
3
2
6
8
Picture 3
1
Hills
4
3
10
9
Picture 4
1
Surf
3
1
3
9
8
10
Frozen
0
0
10
9
11
Feeling of town
0
0
12
10
12
Cliff on water
1
Hills by sea
4
5
13
15
11
13
16
12
14
20
13
15
25
14
Score
0 1 2
3
4
5
6 7 8
9
10
11
12
13
14
# of Concepts
3 1 2 3 1 2 1 1 1
2
3
Table A-5 (continued)
Approved For Release 2006/08/09 LASSLEJE -788R001800050001-8
UNCLASSIFIED
-
1
L
L
-
I
I
L
L
-
-
Weighted Average
2.49
35
TRANSCRIPT EVALUATION
5
6
6
8
PxRxQ
0
0
2

Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
13:13
Target :
Punkharaja
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
3
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(0)
Score
PxRxQ
Score
1
Picture 1
1
Scattered lakes
5
1
5
0
0
2
Land/water interface
1
Scattered lakes
5
4
13
1
1
3
Ridges
1
Islands
4
2
7
4
Small ups and downs
1
Islands
4
4
12
5
Col
1
Location
4
4
12
4
6
Picture 2
1
Islands
4
1
4
5
5
7
Rocky
0
0
36
8
Picture 3
1
Islands
4
1
4
9
8
9
Picture 4
0
0
10
Frozen
1
Location
4
4
12
10
9
11
Feeling of town
1
Town
4
5
13
12
10
12
Cliff on water
0
0
15
11
13
16
12
14
20
13
15
25
14
Score
0
1
2
3
4
5
6 7 8 9
10
11
12
13
14
# of Concepts
3
3
1
4 1
1
3
2
Table A-5 (continued)
Approved For Release 20UNCLASSIFIED0788R001800050001-8
UNCLASSIFIED
1
1
L
1
1
I
I
L
I
L
1
-
2
2
2
3
3
TRANSCRIPT EVALUATION
6
6
7
1
Weighted Average
2.42
1
1

I
1
-
-
-
-
-
L
1
-
L
L
Begin Time:
12:46
Target :
Inverness
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
4
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Role-
vance
(R)
Quality
(Q)
Score
PxRxQ Score
1
Down jagged
0
0
0
0
2
Picture 1
1
Sloping bank
2
1
2
1
1
3
Flat
1
Area
4
3
10
2
2
4
Water
1
River
4
4
12
3
3
5
Green
1
Grass
1
3
3
6
Picture 2
1
River banks
2
1
2
7
Down/up
0
0
8
Deep valley
0
0
9
Picture 3
2
10
Down
1
Banks
2
1
2
11
Land/water interfaces
1
River
4
4
12
12
10
12
Descending
1
Banks
2
1
2
15
11
13
Trees
1
Trees
2
5
9
16
12
14
Winding river
1
River
4
5
13
20
13
15
Jungle
1
Trees
2
1
2
25
14
Score
0 1 2 3
4
5
6789
10
11
12
13
14
# of Concepts
3 5 2 2 2 2
1 1
2
1
Table A-5
(continued)
Approved For Release 2000 MOLASSIFIED0788R001800050001-8
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
UNCLASSIFIED
37
TRANSCRIPT EVALUATION
5
5
6
6
8
7
9
8
10
9
Weighted Average
1.21
4

1
1
1
-
1
I
£
1
L
L
=
£
-
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
Class:
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
(Q)
Score
PxRXQ
Score
1
Down jagged
1
Sloping rocks
3
2
6
0
0
2
Picture 1
H
Sloping rocks
2
1
1
3
Flat
1
Bay
3
1
3
2
2
4
Water
1
Bay
3
4
10
5
Green
0
0
6
Picture 2
1
Sloping rocks
3
1
3
7
Down/up
0
0
0
8
Deep valley
0
0
9
Picture 3
2
9
8
10
Down
1
Sloping rocks
3
1
3
10
9
11
Land/water interfaces
1
Peninsula
5
4
13
12
Descending
1
Sloping rocks
3
1
3
13
Trees
1
Trees
2
4
7
14
Winding river
1
Bay
3
1
3
20
13
15
Jungle
1
Trees
2
3
6
25
14
Score
0
1
2 3 45 6
7 | 8|
9
10
11
12
13
14
# of Concepts
3 5 2 1 3
3 |1
1
1
Table A-5 (continued)
Approved For Release 2000/NJ/CLASSIFIED.788R001800050001-8
UNCLASSIFIED
38
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
12:46
Target :
Hango
Viewer:
#002
B
1
3
3
4
4
5
5
6
6
8
7
12
10
15
11
16
12
Weighted Average
1.38
TRANSCRIPT EVALUATION

Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
12:46
Target :
Fernando de Noronha
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
4
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Rele-
vance
(R)
Quality
Score
Score
1
Down jagged
1
Hills
4
3
10
0
0
2
Picture 1
1
Hills
4
4
12
1
3
Flat
0
0
2
2
4
Water
1
Ocean
4
4
12
3
3
5
Green
1
Vegetation
2
3
6
6
Picture 2
1
Hills
4
1
4
5
5
7
Down/up
1
Hills
4
1
4
8
Deep valley
0
0
9
Picture 3
2
9
8
10
Down
1
Sloping hills
4
1
4
11
Land/water interfaces
1
Island
5
4
13
12
Descending
1
Sloping hills
4
1
4
13
Trees
1
Vegetation
2
5
9
16
12
14
Winding river
1
Ocean
4
2
7
20
13
15
Jungle
1
Vegetation
2
5
9
25
14
Score
0
1
2
3
14
5
6
7
8
9
10
11
12
13
14
# of Concepts
2
4
1 2
3 2 1 1
2
1
2
1
Table A-5 (continued)
Approved For Release 200JNJCLASSIFIED.788R001800050001-8
UNCLASSIFIED
39
L
1
-
=
I
I
L
1
-
TRANSCRIPT EVALUATION
4
4
6
6
8
7
10
9
12
10
15
11
Weighted Average
2.22
PXRxQ
-

Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Begin Time:
12:46
Target :
Punkahar ju
Viewer:
#002
End Time:
Method of Targeting:
Coordinates
Date:
14 December 1981
Length of Session:
Session:
4
Class:
B
Concept
Number
Transcript Concept
D
P
Element of Target
Role-
vance
(R)
Quality
(Q)
Score
PxRxQ
Score
1
Down jagged
0
0
0
0
2
Picture 1
1
Islands
4
3
10
1
1
3
Flat
1
Area
4
3
10
2
2
4
Water
1
Lakes
5
4
13
5
Green
0
0
4
4
6
Picture 2
1
Lake bottoms
5
1
5
7
Down/up
1
Lake bottoms
5
1
5
8
Deep valley
1
Lake bottoms
5
1
5
9
Picture 3
2
9
8
10
Down
1
Lake bottoms
5
1
5
10
9
11
Land/water interfaces
1
Lakes
5
4
13
12
Descending
1
Islands
4
1
4
15
11
13
Trees
1
Trees
2
4
7
.
14
Winding river
1
Connected lakes
5
2
9
20
13
15
Jungle
1
Trees
2
1
2
25
14
Score
0
| 1
2 3
1
4
5
6 7
8
9
10
11
12
13
14
# of Concepts
261234
1
1 |
2
2
Table A-5
(concluded )
Approved For Release 200d/DOL:ASSIFIED00788R001800050001-8
UNCLASSIFIED
40
1
L
L
L
L
-
TRANSCRIPT EVALUATION
3
3
5
5
6
6
8
7
12
10
16
12
Weighted Average
2.21
L
1

-
-
-
-
-
-
Session/Target
Inverness
Hangö
Fernando
de Noronha
Punkaharju
2
4.13
2.72
0.22
2.11
3
1.12
2.65
2.49
2.42
4
1.21
1.38
2.22
2.21
1
1.53
2.52
2.36
0.90
*
Scores computed with non-uniform target relevance factors.
1
1
1
-
=
-
=
-
Approved For Release 20UNCLASSIFIED0788R001800050001-8
(U) Two types of overall assessment were chosen to emphasize the
versatility of the evaluation procedure, (1) a simple rank ordering based
on weighted average scores, and (2) a concept-by-concept, non-parametric,
statistical technique. Table A-6 shows the results of the first method,
the rank ordering. For convenience, the correct matches are underlined.
Table A-6
(U) A RANK ORDERING OF WEIGHTED AVERAGES (U)
-
1
-
(U)
From Table A-6, we see that there were 3 first-place matches
and 1 fourth-place match. The probability of obtaining 3 of 4 possible
first-place matches from chance fluctuations alone are less than 0.051.
The point spread between the best match (Inverness) and the worst match
(Punkaharju) are in qualitative agreement with a subjective "first look"
at the quality of the transcripts as well.
(U) The second analysis determines the significance of the difference
between the correct concept/target matches and a control set of matches.
All concept/target matches that are not the correct matches act as an
internal control set. To avoid any invalid assumptions as to the correct
parent distribution, a non-parametric statistical test, the Mann-Whitney
U-Test, was chosen for the analysis.
4
41
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
*

Approved For Release 2009/09/SdASSLJED0788R001800050001-8
(U)
It is beyond the scope of this report to review the details
of the Mann-Witney U-Test; thus, only the results are quoted here. The
probability that the set of correct concept/target matches is statistically
indistinguishable from the control concept/target matches is less than
0.071.
-
-
(U)
There are a number of additional statistical procedures that
could be used to analyze the results of this evaluation technique. The
two cited above, however, represent a spread in complexity that demonstrates
the internal consistency of the basic evaluation procedure. With only four
similar RV sessions, the evaluation technique nearly reached the 0.05 level
of statistical significance with each of the two statistical procedures,
a result indicating a successful outcome with regard to the overall
assessment procedure.
-
-
-
-
-
42
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8

-
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Appendix B
(U)
SUMMARY OF EVALUATION TECHNIQUE (U)
Step
Action
1
Task coordinator defines the evaluation goal. He/she identifies
target elements and assigns target element relevance factors as
appropriate.
2
Analyst conceptualizes responses and prepares an RV assessment
sheet for each response.
3
Repeated concepts are noted in the "D" column.
4
Copies of the sheets from Item 2 are made; one for each possible
target used in the analysis.
FOR EACH POSSIBLE RESPONSE/TARGET COMBINATION:
5
Identify a target element for each concept not marked in the
"D" column; mark a 1 in the "p" column and write the target
element and its relevance factor from Step 1 in the appropriate
columns. (Write 0 and blanks if no element can be found. )
6
Using Table 3, assign a quality rating for all present (p = 1)
concept/element combinations.
7
Compute the score as follows :
a.
Calculate relevance (R) X quality (Q)
b. Convert R X Q to an integer between 0 and 14 using
the conversion table provided.
8
Enter the number of concepts that obtained each possible score
in the space provided.
9
Calculate the weighted average using:
Az = 0.357 E f VE SIE VE
where :
S. is the score and f. is the number of concepts
that obtained score
j
j = 0, 1, 2, ... , 14
10
For each response, rank order the weighted averages.
43
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
-

Approved For Release 2000JJJ CLASSIFIED. 788R001800050001-8
-
=
-
-
-
-
-
-
-
REFERENCES (U)
1. H. E. Puthoff and R. Targ, "A Perceptual Channel for Information
Transfer over Kilometer Distances: Historical Perspective and
Recent Research," Proc. IEEE, Vol. 64, No. 3, pp. 329-354 (March 1976).
2. C. Scott, "On the Evaluation of Verbal Material in Parapsychology:
A Discussion of Dr. Pratt's Monograph," J. Soc. Psych. Res., Vol. 46,
No. 752, pp. 79-90 (June 1972).
3. R. Targ, H. E. Puthoff, and E. C. May, "State of the Art in Remote
Viewing Studies at SRI," Technical Session on Research in Psycho-
energetics, Proc. 1977 IEEE International Conf. on Cybernetics and
Society, Washington, D.C. (September 20, 1977).
4. S. Siegel, Nonparametric Statistics, pp. 116-120 (McGraww-Hill, 1956).
44
UNCLASSIFIED
Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8