Netty SSL/TLS 的使用与分析

前段时间公司在做的产品需要支持 HTTPS 协议，这篇文章就来分享下通过 Netty 使用 SSL/TLS 以及相关源码分析。

SSL/TLS 的原理就不介绍了，直接上使用教程。

首先，自定义证书：

SelfSignedCertificate ssc = new SelfSignedCertificate();
SslContext sslCtx = SslContextBuilder.forServer(ssc.certificate(), ssc.privateKey()).build();

上面代码直接使用 Netty 自带的自签名证书工具，另外也可以使用 jdk 的 keytool 或 openssl 等工具生成证书。

构造 ChannelInitializer：

public class NettySslHandlerInitializer extends ChannelInitializer<SocketChannel> {

    private final SslContext sslCtx;

    NettySslHandlerInitializer(SslContext sslCtx) {
        this.sslCtx = sslCtx;
    }

    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        // 创建一个新的 SSLEngine 对象
        SSLEngine sslEngine = sslCtx.newEngine(UnpooledByteBufAllocator.DEFAULT);
        // 配置为 server 模式
        sslEngine.setUseClientMode(false);
        // 选择需要启用的 SSL 协议，如 SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2 等
        sslEngine.setEnabledProtocols(sslEngine.getSupportedProtocols());
        // 选择需要启用的 CipherSuite 组合，如 ECDHE-ECDSA-CHACHA20-POLY1305 等
        sslEngine.setEnabledCipherSuites(sslEngine.getSupportedCipherSuites());
        // 添加 SslHandler 之 pipeline 中
        pipeline.addLast("ssl", new SslHandler(sslEngine));

        pipeline.addLast("encoder,", new HttpResponseEncoder());
        pipeline.addLast("decoder", new HttpRequestDecoder());
        pipeline.addLast("deflate", new HttpContentCompressor());
        pipeline.addLast("aggregator", new HttpObjectAggregator(1024*1024*100));
        pipeline.addLast("chunk", new ChunkedWriteHandler());

        pipeline.addLast("demo", new NettySslHandler());

    }
}

SSL 协议各版本参考：维基百科

基本的使用方法很简单，如上述所示。我们来重点分析下 Netty SSL 的内部实现原理。

首先看下 SslContext 是怎么通过 SslContextBuilder 构造的，直接看 SslContextBuilder 的 build 方法即可：

public SslContext build() throws SSLException {
    // 如果是 forServer，则构建 newServerContextInternal
    if (forServer) {
        return SslContext.newServerContextInternal(provider, trustCertCollection,
            trustManagerFactory, keyCertChain, key, keyPassword, keyManagerFactory,
            ciphers, cipherFilter, apn, sessionCacheSize, sessionTimeout, clientAuth, startTls);
    } else {
        // 否则构建 newClientContextInternal
        return SslContext.newClientContextInternal(provider, trustCertCollection,
            trustManagerFactory, keyCertChain, key, keyPassword, keyManagerFactory,
            ciphers, cipherFilter, apn, sessionCacheSize, sessionTimeout);
    }
}

newServerContextInternal 实现如下：

static SslContext newServerContextInternal(
        SslProvider provider,
        X509Certificate[] trustCertCollection, TrustManagerFactory trustManagerFactory,
        X509Certificate[] keyCertChain, PrivateKey key, String keyPassword, KeyManagerFactory keyManagerFactory,
        Iterable<String> ciphers, CipherSuiteFilter cipherFilter, ApplicationProtocolConfig apn,
        long sessionCacheSize, long sessionTimeout, ClientAuth clientAuth, boolean startTls) throws SSLException {

    if (provider == null) {
        provider = defaultServerProvider();
    }

    switch (provider) {
        case JDK:
        // jdk 版本实现
        return new JdkSslServerContext(
            trustCertCollection, trustManagerFactory, keyCertChain, key, keyPassword,
            keyManagerFactory, ciphers, cipherFilter, apn, sessionCacheSize, sessionTimeout,
            clientAuth, startTls);
        case OPENSSL:
        // openssl 版本实现
        return new OpenSslServerContext(
            trustCertCollection, trustManagerFactory, keyCertChain, key, keyPassword,
            keyManagerFactory, ciphers, cipherFilter, apn, sessionCacheSize, sessionTimeout,
            clientAuth, startTls);
    }
}

OpenSSL 版本实现内部基本上采用 native 代码，方便起见，我们直接看 JDK 版本实现。

OpenSslServerContext 的构造很简单，就是基本的初始化赋值操作。

我们在构造 ChannelInitializer 时，将 SslHandler 添加到 pipeline 中，那么就来看下 SslHandler 的 decode 方法：

protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws SSLException {
    final int startOffset = in.readerIndex();
    final int endOffset = in.writerIndex();
    int offset = startOffset;
    int totalLength = 0;

    boolean nonSslRecord = false;

    while (totalLength < OpenSslEngine.MAX_ENCRYPTED_PACKET_LENGTH) {
        final int readableBytes = endOffset - offset;

        // 获取 packet 长度
        final int packetLength = getEncryptedPacketLength(in, offset);
        if (packetLength == -1) {
            nonSslRecord = true;
            break;
        }

        if (packetLength > readableBytes) {
                // wait until the whole packet can be read
            this.packetLength = packetLength;
            break;
        }

        int newTotalLength = totalLength + packetLength;
        if (newTotalLength > OpenSslEngine.MAX_ENCRYPTED_PACKET_LENGTH) {
                // Don't read too much.
            break;
        }

        // We have a whole packet.
        // Increment the offset to handle the next packet.
        offset += packetLength;
        totalLength = newTotalLength;
    }

    if (totalLength > 0) {
        in.skipBytes(totalLength);
        firedChannelRead = unwrap(ctx, in, startOffset, totalLength) || firedChannelRead;
    }

    if (nonSslRecord) {
            // Not an SSL/TLS packet
        NotSslRecordException e = new NotSslRecordException(
            "not an SSL/TLS record: " + ByteBufUtil.hexDump(in));
        in.skipBytes(in.readableBytes());
        setHandshakeFailure(ctx, e);
        ctx.fireExceptionCaught(e);
    }
}

decode 方法主要是获取 packet 长度、解析 packet 内容。

获取 packet 长度自然是通过 getEncryptedPacketLength 方法：

static int getEncryptedPacketLength(ByteBuf buffer, int offset) {
    int packetLength = 0;

        // SSLv3 or TLS - Check ContentType
    boolean tls;
    switch (buffer.getUnsignedByte(offset)) {
        case SSL_CONTENT_TYPE_CHANGE_CIPHER_SPEC:
        case SSL_CONTENT_TYPE_ALERT:
        // SSL 握手 SSL_CONTENT_TYPE_HANDSHAKE = 22
        case SSL_CONTENT_TYPE_HANDSHAKE:
        case SSL_CONTENT_TYPE_APPLICATION_DATA:
            tls = true;
            break;
        default:
            // SSLv2 or bad data
            tls = false;
    }

    if (tls) {
        // SSLv3 or TLS - Check ProtocolVersion
        int majorVersion = buffer.getUnsignedByte(offset + 1);
        if (majorVersion == 3) {
            // SSLv3 or TLS
            packetLength = buffer.getUnsignedShort(offset + 3) + SSL_RECORD_HEADER_LENGTH;
            if (packetLength <= SSL_RECORD_HEADER_LENGTH) {
                // Neither SSLv3 or TLSv1 (i.e. SSLv2 or bad data)
                tls = false;
            }
        } else {
            // Neither SSLv3 or TLSv1 (i.e. SSLv2 or bad data)
            tls = false;
        }
    }

    if (!tls) {
        // SSLv2 or bad data - Check the version
        int headerLength = (buffer.getUnsignedByte(offset) & 0x80) != 0 ? 2 : 3;
        int majorVersion = buffer.getUnsignedByte(offset + headerLength + 1);
        if (majorVersion == 2 || majorVersion == 3) {
            // SSLv2
            if (headerLength == 2) {
                packetLength = (buffer.getShort(offset) & 0x7FFF) + 2;
            } else {
                packetLength = (buffer.getShort(offset) & 0x3FFF) + 3;
            }
            if (packetLength <= headerLength) {
                return -1;
            }
        } else {
            return -1;
        }
    }
    return packetLength;
}

从 getEncryptedPacketLength 方法可以看出，首先会读取 buffer 的第一个字节，判断是否使用的是 SSL 协议，其中 0x16 表示 SSLv3 或 TLS 握手， 另外因为 SSLv2 版第一个字节是 0x87，会特殊再判断一次(阿里云 PTS 服务居然用的还是 SSLv2 握手-_-!)。buffer 的第二个字节定义了 SSL 的主版本号，第三个字节定义了次版本号。

获取 packet 的长度及握手版本号后，就会开始执行握手操作。

在 SslHandler 的 decode 方法中可以看到，在获取 packet 长度后，会执行 unwrap 方法，unwrap 方法内部实际上是调用了 SSLEngineImpl 的 unwrap 方法，SSLEngineImpl.unwrap 会读取 Record 协议内容，流程大致是

SSLEngineImpl.unwrap -> SSLEngineImpl.readNetRecord -> SSLEngineImpl.readRecord -> Handshaker.process_record -> Handshaker.processLoop -> ServerHandshaker.processMessage

void processMessage(byte var1, int var2) throws IOException {
    if(this.state >= var1 && this.state != 16 && var1 != 15) {
        throw new SSLProtocolException("Handshake message sequence violation, state = " + this.state + ", type = " + var1);
    } else {
        switch(var1) {
            case 1: {
                // 与 client 握手
                ClientHello var3 = new ClientHello(this.input, var2);
                this.clientHello(var3);
            }
            break;

            default: {
                throw new SSLProtocolException("Illegal server handshake msg, " + var1);
            }
        }
    }
}

processMessage 很简单，就是构造 ClientHello 对象，完成握手操作。握手操作的完成过程还是要看 ClientHello 的实现：

 private void clientHello(ClientHello var1) throws IOException {

    ServerNameExtension var2 = (ServerNameExtension)var1.extensions.get(ExtensionType.EXT_SERVER_NAME);
    if(!this.sniMatchers.isEmpty() && var2 != null && !var2.isMatched(this.sniMatchers)) {
        this.fatalSE(112, "Unrecognized server name indication");
    }

    boolean var3 = false;
    CipherSuiteList var4 = var1.getCipherSuites();
    if(var4.contains(CipherSuite.C_SCSV)) {
        var3 = true;
        if(this.isInitialHandshake) {
            this.secureRenegotiation = true;
        } else if(this.secureRenegotiation) {
            this.fatalSE(40, "The SCSV is present in a secure renegotiation");
        } else {
            this.fatalSE(40, "The SCSV is present in a insecure renegotiation");
        }
    }

    RenegotiationInfoExtension var5 = (RenegotiationInfoExtension)var1.extensions.get(ExtensionType.EXT_RENEGOTIATION_INFO);
    if(var5 != null) {
        var3 = true;
        if(this.isInitialHandshake) {
            if(!var5.isEmpty()) {
                this.fatalSE(40, "The renegotiation_info field is not empty");
            }

            this.secureRenegotiation = true;
        } else {
            if(!this.secureRenegotiation) {
                this.fatalSE(40, "The renegotiation_info is present in a insecure renegotiation");
            }

            if(!MessageDigest.isEqual(this.clientVerifyData, var5.getRenegotiatedConnection())) {
                this.fatalSE(40, "Incorrect verify data in ClientHello renegotiation_info message");
            }
        }
    } else if(!this.isInitialHandshake && this.secureRenegotiation) {
        this.fatalSE(40, "Inconsistent secure renegotiation indication");
    }

    if(!var3 || !this.secureRenegotiation) {
        if(this.isInitialHandshake) {
            if(!allowLegacyHelloMessages) {
                this.fatalSE(40, "Failed to negotiate the use of secure renegotiation");
            }

            if(debug != null && Debug.isOn("handshake")) {
                System.out.println("Warning: No renegotiation indication in ClientHello, allow legacy ClientHello");
            }
        } else if(!allowUnsafeRenegotiation) {
            if(this.activeProtocolVersion.v >= ProtocolVersion.TLS10.v) {
                this.warningSE(100);
                this.invalidated = true;
                if(this.input.available() > 0) {
                    this.fatalSE(10, "ClientHello followed by an unexpected  handshake message");
                }

                return;
            }

            this.fatalSE(40, "Renegotiation is not allowed");
        } else if(debug != null && Debug.isOn("handshake")) {
            System.out.println("Warning: continue with insecure renegotiation");
        }
    }

    this.input.digestNow();
    ServerHello var6 = new ServerHello();
    this.clientRequestedVersion = var1.protocolVersion;
    ProtocolVersion var7 = this.selectProtocolVersion(this.clientRequestedVersion);
    if(var7 == null || var7.v == ProtocolVersion.SSL20Hello.v) {
        this.fatalSE(40, "Client requested protocol " + this.clientRequestedVersion + " not enabled or not supported");
    }

    // 1. 选定协议版本号
    this.handshakeHash.protocolDetermined(var7);
    this.setVersion(var7);
    var6.protocolVersion = this.protocolVersion;
    this.clnt_random = var1.clnt_random;
    this.svr_random = new RandomCookie(this.sslContext.getSecureRandom());
    var6.svr_random = this.svr_random;
    this.session = null;
    // -- 省略部分 session 重用的代码 --
    if(this.session == null) {

        this.supportedCurves = (SupportedEllipticCurvesExtension)var1.extensions.get(ExtensionType.EXT_ELLIPTIC_CURVES);
        if(this.protocolVersion.v >= ProtocolVersion.TLS12.v) {
            SignatureAlgorithmsExtension var18 = (SignatureAlgorithmsExtension)var1.extensions.get(ExtensionType.EXT_SIGNATURE_ALGORITHMS);
            if(var18 != null) {
                Collection var24 = var18.getSignAlgorithms();
                if(var24 == null || var24.isEmpty()) {
                    throw new SSLHandshakeException("No peer supported signature algorithms");
                }

                Collection var25 = SignatureAndHashAlgorithm.getSupportedAlgorithms(this.algorithmConstraints, var24);
                if(var25.isEmpty()) {
                    throw new SSLHandshakeException("No signature and hash algorithm in common");
                }

                this.setPeerSupportedSignAlgs(var25);
            }
        }

        this.session = new SSLSessionImpl(this.protocolVersion, CipherSuite.C_NULL, this.getLocalSupportedSignAlgs(), this.sslContext.getSecureRandom(), this.getHostAddressSE(), this.getPortSE());
        if(this.protocolVersion.v >= ProtocolVersion.TLS12.v && this.peerSupportedSignAlgs != null) {
            this.session.setPeerSupportedSignatureAlgorithms(this.peerSupportedSignAlgs);
        }

        List var19 = Collections.emptyList();
        if(var2 != null) {
            var19 = var2.getServerNames();
        }

        this.session.setRequestedServerNames(var19);
        this.setHandshakeSessionSE(this.session);
        // 2. 选择使用的 CipherSuite
        this.chooseCipherSuite(var1);
        this.session.setSuite(this.cipherSuite);
        this.session.setLocalPrivateKey(this.privateKey);
    } else {
        this.setHandshakeSessionSE(this.session);
    }

    var6.cipherSuite = this.cipherSuite;
    var6.sessionId = this.session.getSessionId();
    var6.compression_method = this.session.getCompression();

    var6.write(this.output);
    if(this.resumingSession) {
        this.calculateConnectionKeys(this.session.getMasterSecret());
        this.sendChangeCipherAndFinish(false);
    } else {
        // 3. 交换 key
        if(this.keyExchange != KeyExchange.K_KRB5 && this.keyExchange != KeyExchange.K_KRB5_EXPORT) {
            if(this.keyExchange != KeyExchange.K_DH_ANON && this.keyExchange != KeyExchange.K_ECDH_ANON) {
                if(this.certs == null) {
                    throw new RuntimeException("no certificates");
                }

                CertificateMsg var26 = new CertificateMsg(this.certs);
                this.session.setLocalCertificates(this.certs);
                if(debug != null && Debug.isOn("handshake")) {
                    var26.print(System.out);
                }

                var26.write(this.output);
            } else if(this.certs != null) {
                throw new RuntimeException("anonymous keyexchange with certs");
            }
        }

        Object var27;
        switch(null.$SwitchMap$sun$security$ssl$CipherSuite$KeyExchange[this.keyExchange.ordinal()]) {
            case 1:
            case 3:
            case 4:
            var27 = null;
            break;
            case 2:
            if(JsseJce.getRSAKeyLength(this.certs[0].getPublicKey()) > 512) {
                try {
                    var27 = new RSA_ServerKeyExchange(this.tempPublicKey, this.privateKey, this.clnt_random, this.svr_random, this.sslContext.getSecureRandom());
                    this.privateKey = this.tempPrivateKey;
                } catch (GeneralSecurityException var15) {
                    throwSSLException("Error generating RSA server key exchange", var15);
                    var27 = null;
                }
            } else {
                var27 = null;
            }
            break;
            case 5:
            case 6:
            try {
                var27 = new DH_ServerKeyExchange(this.dh, this.privateKey, this.clnt_random.random_bytes, this.svr_random.random_bytes, this.sslContext.getSecureRandom(), this.preferableSignatureAlgorithm, this.protocolVersion);
            } catch (GeneralSecurityException var14) {
                throwSSLException("Error generating DH server key exchange", var14);
                var27 = null;
            }
            break;
            case 7:
            var27 = new DH_ServerKeyExchange(this.dh, this.protocolVersion);
            break;
            case 8:
            case 9:
            var27 = null;
            break;
            case 10:
            case 11:
            case 12:
            try {
                var27 = new ECDH_ServerKeyExchange(this.ecdh, this.privateKey, this.clnt_random.random_bytes, this.svr_random.random_bytes, this.sslContext.getSecureRandom(), this.preferableSignatureAlgorithm, this.protocolVersion);
            } catch (GeneralSecurityException var13) {
                throwSSLException("Error generating ECDH server key exchange", var13);
                var27 = null;
            }
            break;
            default:
            throw new RuntimeException("internal error: " + this.keyExchange);
        }

        if(var27 != null) {
            if(debug != null && Debug.isOn("handshake")) {
                ((ServerKeyExchange)var27).print(System.out);
            }

            ((ServerKeyExchange)var27).write(this.output);
        }

        // 4. 客户端认证
        if(this.doClientAuth != 0 && this.keyExchange != KeyExchange.K_DH_ANON && this.keyExchange != KeyExchange.K_ECDH_ANON && this.keyExchange != KeyExchange.K_KRB5 && this.keyExchange != KeyExchange.K_KRB5_EXPORT) {
            Collection var31 = null;
            if(this.protocolVersion.v >= ProtocolVersion.TLS12.v) {
                var31 = this.getLocalSupportedSignAlgs();
                if(var31.isEmpty()) {
                    throw new SSLHandshakeException("No supported signature algorithm");
                }

                Set var12 = SignatureAndHashAlgorithm.getHashAlgorithmNames(var31);
                if(var12.isEmpty()) {
                    throw new SSLHandshakeException("No supported signature algorithm");
                }
            }

            X509Certificate[] var28 = this.sslContext.getX509TrustManager().getAcceptedIssuers();
            CertificateRequest var29 = new CertificateRequest(var28, this.keyExchange, var31, this.protocolVersion);
            if(debug != null && Debug.isOn("handshake")) {
                var29.print(System.out);
            }

            var29.write(this.output);
        }

        // 5. 握手完成
        ServerHelloDone var30 = new ServerHelloDone();
        var30.write(this.output);
        this.output.flush();
    }
}

握手基本是以下几个步骤：

1. 选定使用的协议版本号
2. 选定使用的加密套件
3. 交换密钥
4. 客户端认证
5. 握手完成

握手完成后就是数据请求流程了，留在以后来分析。