背景 实体类定义属性id为Long类型,但在调用springdataelasticsearch:3。2。10。RELEASE中的putMapping(Class)方法时却被转换成了keyword类型源码 查看putMapping方法,可以发现最终调用最下边的重载方法classElasticsearchRestTemplate{。。。OverridepublicTbooleanputMapping(ClassTclazz){returnputMapping(clazz,buildMapping(clazz));}OverridepublicTbooleanputMapping(ClassTclazz,Objectmapping){returnputMapping(getPersistentEntityFor(clazz)。getIndexName(),getPersistentEntityFor(clazz)。getIndexType(),mapping);}OverridepublicTbooleanputMapping(StringindexName,Stringtype,ClassTclazz){returnputMapping(indexName,type,buildMapping(clazz));}OverridepublicbooleanputMapping(StringindexName,Stringtype,Objectmapping){Assert。notNull(indexName,NoindexdefinedforputMapping());Assert。notNull(type,NotypedefinedforputMapping());PutMappingRequestrequestnewPutMappingRequest(indexName)。type(type);if(mappinginstanceofString){request。source(String。valueOf(mapping),XContentType。JSON);}elseif(mappinginstanceofMap){request。source((Map)mapping);}elseif(mappinginstanceofXContentBuilder){request。source((XContentBuilder)mapping);}try{returnclient。indices()。putMapping(request,RequestOptions。DEFAULT)。isAcknowledged();}catch(IOExceptione){thrownewElasticsearchException(FailedtoputmappingforindexName,e);}}。。。}复制代码 查看buildMapping方法,因为并没有定义外部mappingPath配置文件,所以走最下边的mappingBuilder。buildPropertyMapping(clazz)来进行解析出String类型的json文件abstractclassAbstractElasticsearchTemplate{。。。protectedStringbuildMapping(C?clazz){loadmappingspecifiedinMappingannotationifpresentif(clazz。isAnnotationPresent(Mapping。class)){StringmappingPathclazz。getAnnotation(Mapping。class)。mappingPath();if(!StringUtils。isEmpty(mappingPath)){StringmappingsResourceUtil。readFileFromClasspath(mappingPath);if(!StringUtils。isEmpty(mappings)){}}else{LOGGER。info(mappingPathinMappinghastobedefined。BuildingmappingsusingField);}}buildmappingfromfieldannotationstry{MappingBuildermappingBuildernewMappingBuilder(elasticsearchConverter);returnmappingBuilder。buildPropertyMapping(clazz);}catch(Exceptione){thrownewElasticsearchException(Failedtobuildmappingforclazz。getSimpleName(),e);}}。。。}复制代码 查看buildPropertyMapping方法classMappingBuilder{。。。StringbuildPropertyMapping(C?clazz)throwsIOException{提前解析出一些通用属性,比如indexName,indexType等等ElasticsearchPersistentE?entityelasticsearchConverter。getMappingContext()。getRequiredPersistentEntity(clazz);构造一个json构造器,以indexType开始XContentBuilderbuilderjsonBuilder()。startObject()。startObject(entity。getIndexType());添加dynamictemplateaddDynamicTemplatesMapping(builder,entity);父子文档判断StringparentTypeentity。getParentType();if(hasText(parentType)){builder。startObject(FIELDPARENT)。field(FIELDTYPE,parentType)。endObject();}属性解析开始标志propertiesbuilder。startObject(FIELDPROPERTIES);具体的properties解析,为根对象非nested对象mapEntity(builder,entity,true,,false,FieldType。Auto,null);builder。endObject()FIELDPROPERTIES。endObject()indexType。endObject()rootobject。close();returnbuilder。getOutputStream()。toString();}privatevoidmapEntity(XContentBuilderbuilder,NullableElasticsearchPersistentEntityentity,booleanisRootObject,StringnestedObjectFieldName,booleannestedOrObjectField,FieldTypefieldType,NullableFieldparentFieldAnnotation)throwsIOException{booleanwriteNestedProperties!isRootObject(isAnyPropertyAnnotatedWithField(entity)nestedOrObjectField);if(writeNestedProperties){StringtypenestedOrObjectField?fieldType。toString()。toLowerCase():FieldType。Object。toString()。toLowerCase();builder。startObject(nestedObjectFieldName)。field(FIELDTYPE,type);if(nestedOrObjectFieldFieldType。NestedfieldTypeparentFieldAnnotation!nullparentFieldAnnotation。includeInParent()){builder。field(includeinparent,parentFieldAnnotation。includeInParent());}builder。startObject(FIELDPROPERTIES);}对象字段属性的解析if(entity!null){entity。doWithProperties((PropertyHandlerElasticsearchPersistentProperty)property{try{if(property。isAnnotationPresent(Transient。class)isInIgnoreFields(property,parentFieldAnnotation)){}buildPropertyMapping(builder,isRootObject,property);}catch(IOExceptione){logger。warn(errormappingpropertywithname{},property。getName(),e);}});}if(writeNestedProperties){builder。endObject()。endObject();}}解析每个property的方法privatevoidbuildPropertyMapping(XContentBuilderbuilder,booleanisRootObject,ElasticsearchPersistentPropertyproperty)throwsIOException{if(property。isAnnotationPresent(Mapping。class)){StringmappingPathproperty。getRequiredAnnotation(Mapping。class)。mappingPath();if(!StringUtils。isEmpty(mappingPath)){ClassPathResourcemappingsnewClassPathResource(mappingPath);if(mappings。exists()){builder。rawField(property。getFieldName(),mappings。getInputStream(),XContentType。JSON);}}}geo标识booleanisGeoPointPropertyisGeoPointProperty(property);completion标识booleanisCompletionPropertyisCompletionProperty(property);nestedobject标识booleanisNestedOrObjectPropertyisNestedOrObjectProperty(property);属性上的Field注解FieldfieldAnnotationproperty。findAnnotation(Field。class);if(!isGeoPointProperty!isCompletionPropertyproperty。isEntity()hasRelevantAnnotation(property)){if(fieldAnnotationnull){}I?extendsTypeI?iteratorproperty。getPersistentEntityTypes()。iterator();ElasticsearchPersistentE?persistentEntityiterator。hasNext()?elasticsearchConverter。getMappingContext()。getPersistentEntity(iterator。next()):mapEntity(builder,persistentEntity,false,property。getFieldName(),isNestedOrObjectProperty,fieldAnnotation。type(),fieldAnnotation);if(isNestedOrObjectProperty){}}MultiFieldmultiFieldproperty。findAnnotation(MultiField。class);if(isGeoPointProperty){applyGeoPointFieldMapping(builder,property);}if(isCompletionProperty){CompletionFieldcompletionFieldproperty。findAnnotation(CompletionField。class);applyCompletionFieldMapping(builder,property,completionField);}判断是否为id属性if(isRootObjectfieldAnnotation!nullproperty。isIdProperty()){applyDefaultIdFieldMapping(builder,property);}elseif(multiField!null){addMultiFieldMapping(builder,property,multiField,isNestedOrObjectProperty);}elseif(fieldAnnotation!null){addSingleFieldMapping(builder,property,fieldAnnotation,isNestedOrObjectProperty);}}。。。}复制代码 至此可以看到,只要fieldName为id或document就判定为是id属性,然后将type设置为keyword并且可被索引。疑问到这里解决classSimpleElasticsearchPersistentProperty{。。。privatestaticfinalListStringSUPPORTEDIDPROPERTYNAMESArrays。asList(id,document);publicSimpleElasticsearchPersistentProperty(Propertyproperty,PersistentE?,ElasticsearchPersistentPropertyowner,SimpleTypeHoldersimpleTypeHolder){。。。this。isIdsuper。isIdProperty()SUPPORTEDIDPROPERTYNAMES。contains(getFieldName());。。。}OverridepublicbooleanisIdProperty(){returnisId;}privatevoidapplyDefaultIdFieldMapping(XContentBuilderbuilder,ElasticsearchPersistentPropertyproperty)throwsIOException{builder。startObject(property。getFieldName())。field(FIELDTYPE,TYPEVALUEKEYWORD)。field(FIELDINDEX,true)。endObject();}。。。}复制代码话外题 项目中使用的ElasticSearch实体类都是采取Document指定indexName来操作的,但是索引和表都涉及到分库分表,所以又不能写死,然后就采取的SpEL配合ThreadLocal从上下文里set后get,其实Spring对elasticsearch操作类似于关系型数据库也封装的有一层Repository抽象,名为ElasticsearchRepository,我们可以直接定义实体类操作接口继承就可以完成对单索引的CRUD以及Page等操作,但这样有一个问题,那就是indexName无法动态去调整,所以就放弃了这种,改用更底层的RestHighLevelClient封装的ElasticSearchRestTemplate模版类,这样在面对分库分表时就可以手动去对每个Document进行set不同的indexName,跨索引查询时也可以指定多个,也可以直接指定索引的alias,需要注意的时,在进行更新时,只指定alias是不被允许的,需要手动查出符合条件的Document在进行索引的分组批量更新,即调用ElasticSearch的bulkapi 在对ElasticSearch和数据库的一致性问题上,我是通过封装不同的方法来确保强一致性和最终一致性强一致性 类似插入、更新、删除等场景下,都是放在一个事务里,先操作数据库,再操作ElasticSearch,这样可以确保操作ElasticSearch失败时,数据库可以成功回滚。一般只运用于对数据实时性要求敏感的场景,并且数据量不大的情况,但即便这样还是会有至少1s的延迟,这里就涉及到ElasticSearch的刷盘策略问题上了,这里不展开研究最终一致性 批量的插入、更新这些操作,如果放在一个大事务里,对数据库也是一种压力,所以一般是分批操作数据库,另起一个线程池对事务提交进行监听,将数据库数据同步到ElasticSearch里,在同步成功后反转数据库的同步状态字段。为了确保万无一失,后台会启动一个定时扫描数据库同步字段的线程去定时扫描同步。这种一般适用于大数据量的场景。当然你也可以去监听MySQL的binlog日志来进行同步。