dex中类的加载流程和unexpected DEX异常

在也谈安卓dex的动态加载与MultiDex和热修复中我们讲解了安卓dex文件的加载过程，那么dex文件加载到内存后类的加载流程是怎样的呢？为何被打上预校验的类与其引用的类不在同一个dex中会出现unexpected DEX异常呢？这就需要从安卓中类加载的native层代码寻找答案了。从native代码我们能够更加清楚的明白QQ空间热修复的插桩字节码的原理以及比QQ空间更优的解决办法。

安卓class文件的加载过程及unexpected DEX

前面说过加载一个类起始于ClassLoader中的loadClass方法，关键函数的调用流程如下：

ClassLoader.loadClass->DexPathList.findClass->DexFile.loadClassBinaryName->dalvik_system_DexFile.cpp中Dalvik_dalvik_system_DexFile_defineClassNative
->Class.cpp中的dvmDefineClass->findClassNoInit

/*
* Load the named class (by descriptor) from the specified DEX file.
* Used by class loaders to instantiate a class object from a
* VM-managed DEX.
*/
ClassObject* dvmDefineClass(DvmDex* pDvmDex, const char* descriptor,
        Object* classLoader)
    {
        assert(pDvmDex != NULL);

        return findClassNoInit(descriptor, classLoader, pDvmDex);
    }

可以看到在native层加载类调用的是findClassNoInit函数，这个函数非常长，我们截取其中的关键代码看下这个函数是如何加载一个类的：

static ClassObject* findClassNoInit(const char* descriptor, Object* loader,
        DvmDex* pDvmDex)
    {
       //省略若干代码

       //从缓冲的HashTable中查找该类是否已经加载，若没加载过会返回NULL
        clazz = dvmLookupClass(descriptor, loader, true);
        if (clazz == NULL) {
            const DexClassDef* pClassDef;
            //省略若干细节代码

            if (pDvmDex == NULL) {
                assert(loader == NULL);     /* shouldn't be here otherwise */
                pDvmDex = searchBootPathForClass(descriptor, &pClassDef);
            } else {
                //从dex文件中查找类的定义
                pClassDef = dexFindClass(pDvmDex->pDexFile, descriptor);
            }

            if (pDvmDex == NULL || pClassDef == NULL) {
                if (gDvm.noClassDefFoundErrorObj != NULL) {
                    /* usual case -- use prefabricated object */
                    dvmSetException(self, gDvm.noClassDefFoundErrorObj);
                } else {
                    /* dexopt case -- can't guarantee prefab (core.jar) */
                    dvmThrowNoClassDefFoundError(descriptor);
                }
                goto bail;
            }

            /* found a match, try to load it */
            //根据pClassDef将其转换为ClassObject对象
            clazz = loadClassFromDex(pDvmDex, pClassDef, loader);

            //省略若干代码

            //将查找成功的class添加到HashTable缓冲中，供下次直接查找
            if (!dvmAddClassToHash(clazz)) {
                //省略若干细节代码
            }

            /*
             * Prepare and resolve.
             */
            if (!dvmLinkClass(clazz)) {
                //省略若干细节代码
            }

            //省略若干代码

        }

        //省略若干代码
        bail:
        if (profilerNotified)
            dvmMethodTraceClassPrepEnd();
        assert(clazz != NULL || dvmCheckException(self));
        return clazz;
    }

可以看到findClassNoInit首先会从缓存的已加载了的类的HashTable中去查找该类是否已经加载过，如果已经加载过直接返回该类，否则会从dex中查找该类，如果找到会将其转换为ClassObject对象（对应java层的Class），然后将其添加到HashTable中进行缓存，然后会调用dvmLinkClass函数链接该类，而在链接的过程中调用了Resolve.cpp中的dvmResolveClass函数来解析class，在这个函数中会出现QQ空间热修复提到的预校验问题，因为这个函数比较重要而且函数也不长，因此看下其完整的代码：

ClassObject* dvmResolveClass(const ClassObject* referrer, u4 classIdx,
        bool fromUnverifiedConstant)
    {
        DvmDex* pDvmDex = referrer->pDvmDex;
        ClassObject* resClass;
        const char* className;

        /*
         * Check the table first -- this gets called from the other "resolve"
         * methods.
         */
        resClass = dvmDexGetResolvedClass(pDvmDex, classIdx);
        if (resClass != NULL)
            return resClass;

        LOGVV("--- resolving class %u (referrer=%s cl=%p)",
                classIdx, referrer->descriptor, referrer->classLoader);

        /*
         * Class hasn't been loaded yet, or is in the process of being loaded
         * and initialized now.  Try to get a copy.  If we find one, put the
         * pointer in the DexTypeId.  There isn't a race condition here --
         * 32-bit writes are guaranteed atomic on all target platforms.  Worst
         * case we have two threads storing the same value.
         *
         * If this is an array class, we'll generate it here.
         */
        className = dexStringByTypeIdx(pDvmDex->pDexFile, classIdx);
        if (className[0] != '\0' && className[1] == '\0') {
            /* primitive type */
            resClass = dvmFindPrimitiveClass(className[0]);
        } else {
            resClass = dvmFindClassNoInit(className, referrer->classLoader);
        }

        if (resClass != NULL) {
            /*
             * If the referrer was pre-verified, the resolved class must come
             * from the same DEX or from a bootstrap class.  The pre-verifier
             * makes assumptions that could be invalidated by a wacky class
             * loader.  (See the notes at the top of oo/Class.c.)
             *
             * The verifier does *not* fail a class for using a const-class
             * or instance-of instruction referring to an unresolveable class,
             * because the result of the instruction is simply a Class object
             * or boolean -- there's no need to resolve the class object during
             * verification.  Instance field and virtual method accesses can
             * break dangerously if we get the wrong class, but const-class and
             * instance-of are only interesting at execution time.  So, if we
             * we got here as part of executing one of the "unverified class"
             * instructions, we skip the additional check.
             *
             * Ditto for class references from annotations and exception
             * handler lists.
             */
            if (!fromUnverifiedConstant &&
                    IS_CLASS_FLAG_SET(referrer, CLASS_ISPREVERIFIED))
            {
                ClassObject* resClassCheck = resClass;
                if (dvmIsArrayClass(resClassCheck))
                    resClassCheck = resClassCheck->elementClass;

                if (referrer->pDvmDex != resClassCheck->pDvmDex &&
                    resClassCheck->classLoader != NULL)
                {
                    ALOGW("Class resolved by unexpected DEX:"
                            " %s(%p):%p ref [%s] %s(%p):%p",
                            referrer->descriptor, referrer->classLoader,
                            referrer->pDvmDex,
                            resClass->descriptor, resClassCheck->descriptor,
                            resClassCheck->classLoader, resClassCheck->pDvmDex);
                    ALOGW("(%s had used a different %s during pre-verification)",
                            referrer->descriptor, resClass->descriptor);
                    dvmThrowIllegalAccessError(
                            "Class ref in pre-verified class resolved to unexpected "
                            "implementation");
                    return NULL;
                }
            }

            LOGVV("##### +ResolveClass(%s): referrer=%s dex=%p ldr=%p ref=%d",
                    resClass->descriptor, referrer->descriptor, referrer->pDvmDex,
                    referrer->classLoader, classIdx);

            /*
             * Add what we found to the list so we can skip the class search
             * next time through.
             *
             * TODO: should we be doing this when fromUnverifiedConstant==true?
             * (see comments at top of oo/Class.c)
             */
            dvmDexSetResolvedClass(pDvmDex, classIdx, resClass);
        } else {
            /* not found, exception should be raised */
            LOGVV("Class not found: %s",
                    dexStringByTypeIdx(pDvmDex->pDexFile, classIdx));
            assert(dvmCheckException(dvmThreadSelf()));
        }

        return resClass;
    }

可以从源代码的if语句中提炼出导致抛出unexpected DEX异常需要同时满足以下3个条件（其实是4个条件，只不过因为resClassCheck->classLoader != NULL是恒成立的，所以只能考虑其余3个条件），如下图所示

可以看到只有当3个条件同时满足的时候才会抛出unexpected DEX异常，出现QQ空间热修复中提到的pre-verified问题。换而言之，只要我们破坏这三个条件中的某一个即可防止该异常抛出，我们来看下这三个条件成立的情况：

1. !fromUnverifiedConstant，即fromUnverifiedConstant==false，而这个fromUnverifiedConstant即是调用dvmResolveClass函数时传入的一个bool参数
2. IS_CLASS_FLAG_SET(referrer, CLASS_ISPREVERIFIED)，加载的类的flag是CLASS_ISPREVERIFIED
3. referrer->pDvmDex != resClassCheck->pDvmDex，加载的类和被引用类不在同一个dex中

再来看看如何破坏这3个条件，一个一个来看：

*"fromUnverifiedConstant" should only be set if this call is the direct
* result of executing a "const-class" or "instance-of" instruction, which
* use class constants not resolved by the bytecode verifier.

System.out.print(com.example.fixbug.class);

System.out.print(com.example.fixbug.class);

if(com.example.fixbug.class instanceof Object) ;

* If the referrer was pre-verified, the resolved class must come
* from the same DEX or from a bootstrap class.  The pre-verifier
* makes assumptions that could be invalidated by a wacky class
* loader.  (See the notes at the top of oo/Class.c.)

总结

解决某个被打上CLASS_ISPREVERIFIED预校验标识导致unExpected异常包含3种解决思路：

1. 以QQ空间为代表的防止类被打上该标志，通过字节码插桩方式让原dex中的每个类引用一个与其不在同一个dex中的某个类，这样原dex中的每一个类都不会被打上CLASS_ISPREVERIFIED标志
2. 以Tinker为代表的让打上该标志的类和其引用的所有的类都在同一个dex中，具体做法是让补丁类与原dex进行合成为新的dex，这样即使某个被打上CLASS_ISPREVERIFIED标志的类引用了该补丁类，它们也仍然在同一个dex中
3. 通过让补丁类触发"const-class" / "instance-of"指令来达到让fromUnverifiedConstant标志为true的目的，从而避免预校验过程

3种方案的比较

首先QQ空间这种方式是不大推荐的，因为QQ空间实际上是破坏了执行dexopt阶段时类被打上CLASS_ISPREVERIFIED标识，而dalvik虚拟机之所以要在dexopt阶段就将这些类打上该标识是为了提高运行效率，但是如果在dexopt阶段阻止某些本应该被打上CLASS_ISPREVERIFIED/CLASS_ISOPTIMIZED标志的类被打上该标识， 那么这些类的Verify和Optimize都将在类的初始化阶段进行，很显然在类的初始化阶段去执行这些操作会降低运行效率，因为dexopt操作是在apk安装时进行的，这会将verify和optimize从安装时改到了运行时进行，用图表示如下：

图片引用自https://yq.aliyun.com/articles/222892?utm_content=m_32258#

另外也可以看到QQ空间的插桩做法实际上是对未出现bug前的原dex进行处理，因为无法预计哪个类可能会出现bug，所以需要对原dex中的每个类都插入一段引用另外一个单独dex中类的代码片段。因此最坏的情况就是无出现bug的类，此时插桩就失去了意义还会白白损耗性能。

而Tinker以及方案3属于对出现bug之后的处理，Tinker将出现bug的类修复后与原dex进行合成，方案3通过让出现bug的类修复后主动触发"const-class" / "instance-of"指令。

从操作过程来看，QQ空间方案属于预防，在发布的原包dex的每个类中插入一段引用另外一个单独dex中类的代码片段从而达到每个类都不被打上CLASS_ISOPTIMIZED标志，以避免出现bug后热修复时预校验问题。属于未雨绸缪，不管下不下雨，随时带把伞使得万一下雨时有伞可用。Tinker和方案3属于补救，Tinker出现bug修复后，让修复的类和原dex合成为全新的dex，不规避打上CLASS_ISOPTIMIZED，而是避免打上了该标志的类与其直接引用的所有的类不在同一个dex中这一问题。方案3通过让补丁类主动触发"const-class" / "instance-of"指令避免补丁类进行预校验。很显然出现bug后补救的方案更好，因为预防在很多情况下是不必要的，就像可能绝大多数情况下都不会下雨，比如前面说过最坏的情况就是无bug类出现此时插桩只会白白损耗性能。而补救是在出现bug之后，已经明确知道出现bug了。