Simulation Results

This chapter presents the results from the simulations. The results are presented for all 5 simulated schemes in a separate table for each environment. Note that the simulations demonstrate increased quality as the timespan of the scheme, the size of the scheme and the certificate validation rate increases. This is because a higher number of revocation requests is more likely to follow a certain patter.

Environment

The thesis presented three environments:

Environment Size Validitation rate Validity Period Revocation Rate

1 100 5/day 2 weeks 10%/year

2 10000 25/day daily 10%/year

3 1000000 100/day hourly 10%/year

Environment	Size	Validitation rate	Validity Period	Revocation Rate
1	100	5/day	2 weeks	10%/year
2	10000	25/day	daily	10%/year
3	1000000	100/day	hourly	10%/year

These schemes are not simulated directly, as some of the high values are very performance demanding in the simulations. The simulations become particularly slow and memory demanding for a high timespan, size and/or validation rate. Also, in order to properly simulate a scheme, the timespan can not be less than the validity period.

Because of these restrictions, a set of new environments are defined. Some of these are similar or equivalent to the above environments, wheras others are designed for the purpose of this simulation.

Environment Size Validitation rate Validity Period Revocation Rate

A 100 5/day 2 weeks 10%/year

B 1000 100/day 60 min 10%/year

C 10000 25/day 30 min 10%/year

Environment	Size	Validitation rate	Validity Period	Revocation Rate
A	100	5/day	2 weeks	10%/year
B	1000	100/day	60 min	10%/year
C	10000	25/day	30 min	10%/year

For the simulations below, there are 4 Distribution Points when Distribution Points are used, and 12 delta CRL periods when delta CRLs are used.

Simulation Results

Environment A

Max Request Rate Max Delta Req. Rate Max Network Load Max Processing Load Max Delay

CRL 0.03rq/s 0rq/s 71.73b/s 0.03un/s 1.22ms

CRL DP 0.07rq/s 0rq/s 107.47b/s 0.07un/s 1.16ms

Delta CRL 0.05rq/s 0.05rq/s 140.64b/s 0.05un/s 1.22ms

Delta CRL DP 0.05rq/s 0.07rq/s 118.48b/s 0.07un/s 1.16ms

OCSP 0.07rq/s 0rq/s 66.67b/s 2.87un/s 43.1ms

	Max Request Rate	Max Delta Req. Rate	Max Network Load	Max Processing Load	Max Delay
CRL	0.03rq/s	0rq/s	71.73b/s	0.03un/s	1.22ms
CRL DP	0.07rq/s	0rq/s	107.47b/s	0.07un/s	1.16ms
Delta CRL	0.05rq/s	0.05rq/s	140.64b/s	0.05un/s	1.22ms
Delta CRL DP	0.05rq/s	0.07rq/s	118.48b/s	0.07un/s	1.16ms
OCSP	0.07rq/s	0rq/s	66.67b/s	2.87un/s	43.1ms

Environment B

	Max Request Rate	Max Delta Req. Rate	Max Network Load	Max Processing Load	Max Delay
CRL	1.12rq/s	0rq/s	9,639.07b/s	1.12un/s	1.86ms
CRL DP	1.23rq/s	0rq/s	3,986.13b/s	1.23un/s	1.32ms
Delta CRL	1.27rq/s	1.3rq/s	10,933.87b/s	1.87un/s	1.86ms
Delta CRL DP	1.35rq/s	1.42rq/s	5,232.06b/s	2.28un/s	1.32ms
OCSP	1.47rq/s	0rq/s	1,466.67b/s	63.07un/s	43.1ms

Environment C

	Max Request Rate	Max Delta Req. Rate	Max Network Load	Max Processing Load	Max Delay
CRL	3.25rq/s	0rq/s	238,654b/s	3.25un/s	8.34ms
CRL DP	3.12rq/s	0rq/s	60,563.07b/s	3.12un/s	2.94ms
Delta CRL	3.42rq/s	3.42rq/s	256,487.39b/s	6.83un/s	8.34ms
Delta CRL DP	3.77rq/s	3.77rq/s	78,781.23b/s	7.53un/s	2.94ms
OCSP	3.43rq/s	0rq/s	3,433.33b/s	147.63un/s	43.1ms

Analysis of the simulation runs

Simulation runs have demonstrated that the schemes do not demonstrate any particular pattern for a small number of requests (e.g. with timespan=60s, size=100 and validationRate=25). However, as the number of requests increases, all the schemes starts to exhibit the qualitative behavior described in Cooper's papers as outlined in the thesis.

From the simulation runs listed above, the following can be observed:

There are differences in the maximum request rate. These are, however, not large, and they are likely to be caused by the randomness of the simulation.
The same as for the maximum request rate applies to the maximum delta request rate. Obviously, the delta request rate for CRL, CRL DP and OCSP are 0.
The maximum network load varies widely, but Environment A does not have the same properties as Environment B and C. Since Environment A has a very small size and a low validation rate we choose to study Environment B and C. For both of these, CRL has a high maximum network load that is significantly reduced (up to 75%) by using CRL Distribution Points. The netwok load for Delta CRL is slightly higher (about 10%) than CRL. The reason why the difference is not larger is that the Delta CRLs tend to be very small compared to the full CRLs. By using Delta CRLs with Distribution Points, the timeliness of Delta CRLs can be obtained with a reduced maximum network load not much higher than for CRL Distribution Points. OCSP demonstrates a significantly smaller maximum network load than any of the other schemes.
The simulation runs show that CRL and CRL DP have similar maximum processing loads, although is seems that CRL DP has a slightly higher maximum load. The processing loads for the Delta CRL scheme and teh Delta CRL scheme with DPs are higher (about twice as big). This is because both CRLs and Delta CRLs are processed at the same time. The processing load for OCSP is significantly higher than any of the other scheme because the OCSP responder has to sign every response.
Since the maximum network load never exceeds the bandwidth (10Mbit) and the maximum processing load never exceeds the processing capacity (1000un/s), there is no delay due to queueing in these simulation runs. Consequently, the delay is determined by the transmission time (decided by revocation information size) and processing time. As can be seen above, CRLs and Delta CRLs have the same delay, whereas the delay for the schemes with Distribution Points is slightly smaller due to reduced revocation information size. Because of the high processing load involved in digitally signing the responses, OCSP has a much higher delay.

Analysis of Request Rate graphs

For parameters that involve a sufficient number of certificate validations (as in Environment B and C), the request rate graphs confirms Coope's models qualitatively:

CRL: The maximum request rate for CRL is close to time 0, and it decreases exponentially.
CRL DP: The graph is similar to the graph for CRL, except that it decreases slower.
Delta CRL: Each Delta CRL demonstrates the same properties as a base CRL issuance. The base CRL is unchanged compared to CRL.
Delta CRL w. DP: Same properties as Delta CRL, except that the request rates for both base CRLs and Delta CRLs decrease slower.
OCSP: The request rate is relatively stable, neither increasing nor decreasing.

Conclusion

These simulations have demonstrated that Cooper's models qualitatively describes the revocation schemes studied in this simulation. Also, these simulations show that using Delta CRLs combined with Distribution Points is a good way of reducing the network load and delay while gaining the good timeliness of delta CRLs. It has been shown that OCSP is very attractive when it comes to maximum network load, whereas its high processing load involves a significant delay. The processing load for Environment C is only about 7 times smaller than the processing capacity of a host. In a scheme with more requests or if the processing time for each OCSP response is larger than 43 ms (which represents the signing time for RSA), the requests will have to be queued, further increasing the delay.

A problem with the simulations was the limitations that occured due to simulation time and memory usage. While very useful when designing the simulation, an object-oriented simulation (in Java) may not be the most efficient approach. The simulations could probably have been much more efficient if implemented in a procedural language (e.g. C) or in a mathematical modelling package (e.g. Matlab).

Further Work

A natural extension of this project is to expand the simulation program to include more scheme. In particular, it would be interesting to add Over-Issuing to the program. Also, it would be interesting to study CRTs in a simulation. The program was designed to be expandable, so the expansions should not be to difficult.

The focus of this project was to develop the simulation software and to perform some experimentation. With more time, more simulations could have been performed and analyzed statistically. Also, the results could be used to verify Cooper's models quantitatively.

andrearn@pvv.ntnu.no

Last modified: Wed Apr 26 11:46:31 CEST 2000