xiaocainiaoya's blog

日常积累

0%

shardingsphere与mybatisflex冲突

发表于 分类于

​ 最近在看shardingsphere分表相关实现，做了一个示例，将shardingsphere引入到之前一直用的以springboot3为基础的ruoyi-flex项目，引入之后，启动一直报一个错误。

问题

Caused by: java.lang.NullPointerException: Properties strategy can not be null when uses class based sharding strategy.
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:1010)
at org.apache.shardingsphere.sharding.algorithm.sharding.classbased.ClassBasedShardingAlgorithm.getStrategy(ClassBasedShardingAlgorithm.java:66)
at org.apache.shardingsphere.sharding.algorithm.sharding.classbased.ClassBasedShardingAlgorithm.init(ClassBasedShardingAlgorithm.java:59)
at org.apache.shardingsphere.infra.util.spi.type.typed.TypedSPIRegistry.findRegisteredService(TypedSPIRegistry.java:66)
at org.apache.shardingsphere.infra.util.spi.type.typed.TypedSPIRegistry.getRegisteredService(TypedSPIRegistry.java:113)
at org.apache.shardingsphere.infra.config.algorithm.ShardingSphereAlgorithmFactory.createAlgorithm(ShardingSphereAlgorithmFactory.java:40)
at org.apache.shardingsphere.sharding.factory.ShardingAlgorithmFactory.newInstance(ShardingAlgorithmFactory.java:45)
at org.apache.shardingsphere.sharding.rule.ShardingRule.lambda$new$0(ShardingRule.java:120)
at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:986)
at org.apache.shardingsphere.sharding.rule.ShardingRule.(ShardingRule.java:120)
at org.apache.shardingsphere.sharding.rule.builder.ShardingRuleBuilder.build(ShardingRuleBuilder.java:41)
at org.apache.shardingsphere.sharding.rule.builder.ShardingRuleBuilder.build(ShardingRuleBuilder.java:35)
at org.apache.shardingsphere.infra.rule.builder.database.DatabaseRulesBuilder.build(DatabaseRulesBuilder.java:58)
at org.apache.shardingsphere.infra.metadata.database.ShardingSphereDatabase.create(ShardingSphereDatabase.java:87)
at org.apache.shardingsphere.infra.metadata.database.ShardingSphereDatabasesFactory.createGenericDatabases(ShardingSphereDatabasesFactory.java:81)
at org.apache.shardingsphere.infra.metadata.database.ShardingSphereDatabasesFactory.create(ShardingSphereDatabasesFactory.java:69)
at org.apache.shardingsphere.mode.metadata.MetaDataContextsFactory.create(MetaDataContextsFactory.java:91)
at org.apache.shardingsphere.mode.metadata.MetaDataContextsFactory.create(MetaDataContextsFactory.java:69)
at org.apache.shardingsphere.mode.manager.standalone.StandaloneContextManagerBuilder.build(StandaloneContextManagerBuilder.java:49)
at org.apache.shardingsphere.driver.jdbc.core.datasource.ShardingSphereDataSource.createContextManager(ShardingSphereDataSource.java:76)
at org.apache.shardingsphere.driver.jdbc.core.datasource.ShardingSphereDataSource.(ShardingSphereDataSource.java:64)
at org.apache.shardingsphere.driver.api.ShardingSphereDataSourceFactory.createDataSource(ShardingSphereDataSourceFactory.java:93)
at org.apache.shardingsphere.spring.boot.ShardingSphereAutoConfiguration.shardingSphereDataSource(ShardingSphereAutoConfiguration.java:91)
at org.apache.shardingsphere.spring.boot.ShardingSphereAutoConfiguration$$SpringCGLIB$$0.CGLIB$shardingSphereDataSource$3()
at org.apache.shardingsphere.spring.boot.ShardingSphereAutoConfiguration$$SpringCGLIB$$FastClass$$1.invoke()
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:258)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
at org.apache.shardingsphere.spring.boot.ShardingSphereAutoConfiguration$$SpringCGLIB$$0.shardingSphereDataSource()
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:140)
… 44 common frames omitted

断点打在报错点之后，向上跟踪堆栈后发现，是由于创建shardingsphereDataSource时取用的ruleConfiguration中无法获取到strategy。从断点可知，几乎所有字段都是空值。

shardingsphere.png

所以近一步就去排查为什么参数ObjectProvider<List<RuleConfiguration>> rules里的这些参数为空。

1
2
3
4
5
6
7
@Bean
@Conditional(LocalRulesCondition.class)
@Autowired(required = false)
public DataSource shardingSphereDataSource(final ObjectProvider<List<RuleConfiguration>> rules, final ObjectProvider<ModeConfiguration> modeConfig) throws SQLException {
    Collection<RuleConfiguration> ruleConfigs = Optional.ofNullable(rules.getIfAvailable()).orElseGet(Collections::emptyList);
    return ShardingSphereDataSourceFactory.createDataSource(databaseName, modeConfig.getIfAvailable(), dataSourceMap, ruleConfigs, props.getProps());
}

跟踪shardingsphereDataSource的bean创建流程，发现这块确实的数据是通过注入对象shardingRuleConfiguration中获得，所以再次跟踪shardingRuleConfiguration的bean创建流程。

它的初始化方法在：ShardingRuleSpringBootConfiguration配置类中，这里通过ObjectProvider注入了Map<String, ShardingAlgorithm>对象。

1
2
3
4
5
6
7
8
9
10
11
12
13
@Bean
public RuleConfiguration shardingRuleConfiguration(final ObjectProvider<Map<String, ShardingAlgorithm>> shardingAlgorithmProvider,
                                                   final ObjectProvider<Map<String, KeyGenerateAlgorithm>> keyGenerateAlgorithmProvider,
                                                   final ObjectProvider<Map<String, ShardingAuditAlgorithm>> shardingAuditAlgorithmProvider) {
    Map<String, ShardingAlgorithm> shardingAlgorithmMap = Optional.ofNullable(shardingAlgorithmProvider.getIfAvailable()).orElse(Collections.emptyMap());
    Map<String, KeyGenerateAlgorithm> keyGenerateAlgorithmMap = Optional.ofNullable(keyGenerateAlgorithmProvider.getIfAvailable()).orElse(Collections.emptyMap());
    Map<String, ShardingAuditAlgorithm> shardingAuditAlgorithmMap = Optional.ofNullable(shardingAuditAlgorithmProvider.getIfAvailable()).orElse(Collections.emptyMap());
    AlgorithmProvidedShardingRuleConfiguration result = swapper.swapToObject(yamlConfig.getSharding());
    result.setShardingAlgorithms(shardingAlgorithmMap);
    result.setKeyGenerators(keyGenerateAlgorithmMap);
    result.setAuditors(shardingAuditAlgorithmMap);
    return result;
}

org.springframework.beans.factory.ObjectProvider 是 Spring 框架提供的一个功能接口，主要用于依赖注入时的灵活获取 Bean，它是对传统 BeanFactory.getBean() 方法的增强，提供了更安全、更便捷的 Bean 查找和注入方式。如果获取不到，它会返回null，不会抛异常NoSuchBeanDefinitionException

进入shardingAlgorithmProvider.getIfAvailable()方法，实际上它也是springbean的创建流程，它尝试获取在spring容器中的ShardingAlgorithm的实现类，通过断点跟踪发现，这里从容器中获取到的shardingAlgorithm对象的相关参数就是空的。

到这里，简单复个盘：

报错的原因是创建shardingsphereDataSource时，其中的algorithm相关参数为空→ 跟踪shardingRuleConfiguration，发现是从ObjectProvider<Map<String, ShardingAlgorithm>>来的 →跟踪shardingAlgorithmProvider.getIfAvailable()发现，这时从容器中获取到的ShardingAlgorithm实现就是空的。

所以现在就变成了新的问题：为什么从容器中获取到的ShardingAlgorithm对象，值都是空的？

创建一个新的工程，不包含mybatis-flex，观察ShardingAlgorithm的创建流程，通过跟踪发现，ShardingAlgorithm对象的值的填充时机，是在后置接口ShardingAlgorithmProvidedBeanRegistry中，当ShardingAlgorithmProvidedBeanRegistry后置接口方法触发时，才会将相关配置项的值填充到ShardingAlgorithm对象中。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
protected final void registerBean(final String prefix, final Class<T> algorithmClass, final BeanDefinitionRegistry registry) {
    if (!PropertyUtil.containPropertyPrefix(environment, prefix)) {
        return;
    }
    Map<String, Object> parameterMap = PropertyUtil.handle(environment, prefix, Map.class);
    Collection<String> algorithmNames = parameterMap.keySet().stream().map(key -> key.contains(POINT) ? key.substring(0, key.indexOf(POINT)) : key).collect(Collectors.toSet());
    Map<String, AlgorithmConfiguration> algorithmConfigs = createAlgorithmConfigurations(prefix, algorithmNames);
    ShardingSphereServiceLoader.register(algorithmClass);
    // 把相关配置项填充到当前对象中
    for (Entry<String, AlgorithmConfiguration> entry : algorithmConfigs.entrySet()) {
        AlgorithmConfiguration algorithmConfig = entry.getValue();
        BeanDefinitionBuilder builder = BeanDefinitionBuilder.genericBeanDefinition(ShardingSphereAlgorithmFactory.createAlgorithm(algorithmConfig, algorithmClass).getClass());
        registry.registerBeanDefinition(entry.getKey(), builder.getBeanDefinition());
        propsMap.put(entry.getKey(), algorithmConfig.getProps());
    }
}

所以跟踪原工程的PostProcessorRegistrationDelegate类，观察后置处理接口的执行情况，果然发现了一个不一样的后置处理器JobRegistryBeanPostProcessor，这个后置处理器在ShardingAlgorithmProvidedBeanRegistry之前，也就是说，一个bean会先经过job的这个后置处理，再经过shardingAlgorithm后置处理器。

继续跟踪JobRegistryBeanPostProcessor，在注册JobRegistryBeanPostProcessor置后置处理器列表中时，触发了它的初始化类BatchAutoConfiguration#SpringBootBatchConfiguration，而它的参数中包含了DataSource dataSource。

综上：因为JobRegistryBeanPostProcessor比ShardingAlgorithmProvidedBeanRegistry在后置处理器列表靠前，导致JobRegistryBeanPostProcessor先进入初始化流程，而它需要dataSource，则进入了dataSource的初始化流程，而dataSource的初始化会触发到shardingsphereDataSource初始化，而shardingsphereDataSource初始化会获取ShardingAlgorithm，这时因为ShardingAlgorithmProvidedBeanRegistry还没执行，导致相关值还没填充，导致报错。

解决

查询相关文档发现，官方貌似也发现存在这个问题，JobRegistryBeanPostProcessor注入的时机过早会导致一些问题，所以官方注明了替代方案JobRegistrySmartInitializingSingleton

但是在工程里，JobRegistryBeanPostProcessor是在spring-batch-core包中的类DefaultBatchConfiguration声明

1
2
3
4
5
6
7
8
9
10
11
12
@Bean
public JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor() throws BatchConfigurationException {
	JobRegistryBeanPostProcessor jobRegistryBeanPostProcessor = new JobRegistryBeanPostProcessor();
	jobRegistryBeanPostProcessor.setJobRegistry(jobRegistry());
	try {
		jobRegistryBeanPostProcessor.afterPropertiesSet();
		return jobRegistryBeanPostProcessor;
	}
	catch (Exception e) {
		throw new BatchConfigurationException("Unable to configure the default job registry BeanPostProcessor", e);
	}
}

所以，解决方案就是覆盖这个类，并对这部分代码注释，不声明JobRegistryBeanPostProcessor这个bean，同事声明替代方案JobRegistrySmartInitializingSingleton。

1
2
3
4
5
6
7
8
/**
	* 注册JobRegistrySmartInitializingSingleton（关键：不使用JobRegistryBeanPostProcessor）
	*/
@Bean
public JobRegistrySmartInitializingSingleton jobRegistrySmartInitializingSingleton() {
	// 初始化并设置参数
	return new JobRegistrySmartInitializingSingleton(jobRegistry);
}
-------------本文结束感谢您的阅读-------------