46 Commits

Author SHA1 Message Date
magese a3705f5753 Merge remote-tracking branch 'origin/master' 2019-07-11 15:08:40 +08:00
magese cc9f0d7bea Update lucene to 8.1.1 2019-07-11 13:51:05 +08:00
Magese 7a2515a134 Update README.md
Add HitCount
2019-05-29 08:58:17 +08:00
magese 2ebf31fed5 Update README.md 2019-05-27 13:23:53 +08:00
magese fd35153cce Update lucene to 8.1.0 2019-05-27 10:23:32 +08:00
magese 48a0234bb6 更新搜狗2019流行词汇; 2019-05-27 10:21:00 +08:00
magese a00336d239 Update README.md 2019-05-15 13:28:47 +08:00
magese 32d874b1ba Merge remote-tracking branch 'origin/master' 2019-05-15 10:38:25 +08:00
magese 7b0d009d32 support solr 8.0.0 2019-05-15 10:37:29 +08:00
Magese e3695f9192 Create CONTRIBUTING.md 2019-03-01 15:02:32 +08:00
Magese 33ac83d619 Update issue templates 2019-03-01 15:01:25 +08:00
magese fa1451042b Update README.md 2019-03-01 09:08:22 +08:00
magese c2e408473a Update Lucene version to 7.7.1 2019-02-28 09:46:10 +08:00
magese 712da9e7fc UPDATE README.md 2019-02-15 17:22:17 +08:00
magese 76a4499777 UPDATE README.md 2019-02-15 17:20:53 +08:00
magese e6023b1317 Update lucene version to 7.7.0 2019-02-15 14:53:02 +08:00
magese 2b2f551f0e UPDATE LUCENE VERSION TO 7.7.0 2019-02-15 14:39:30 +08:00
Magese 39a4fe434f Update README.md 2019-02-13 11:19:42 +08:00
magese c73e99133a ADD THANKS TO IDEA 2019-01-18 09:42:08 +08:00
Magese 1dc338cbba Update README.md 2019-01-10 16:54:31 +08:00
Magese 76c5092fd9 Update README-CLOUD.md 2018-12-26 17:35:34 +08:00
magese e0d799605b UPDATE README.md 2018-12-26 17:31:18 +08:00
magese ac74bac851 UPDATE README.md 2018-12-26 15:26:40 +08:00
magese c42c6c0739 UPDATE README.md 2018-12-26 15:19:04 +08:00
magese 4b692bcb13 UPDATE README.md 2018-12-26 15:15:06 +08:00
magese f4cf971956 ADD SOLR-CLOUD README.md 2018-12-26 15:14:59 +08:00
magese d1481b0cd7 README IMG 2018-12-26 15:14:23 +08:00
magese 2a61cd2469 Update lucene version to 7.6.0 2018-12-26 14:05:54 +08:00
magese 98ee5afe64 修改读取文件逻辑优化效率,新增词典文件注释功能,#开头的词语将不会进行加载。 2018-12-26 14:05:32 +08:00
magese 720278edd1 增加配置文件名称读取,更换使用的IO流。 2018-12-26 14:03:37 +08:00
magese bac4768ed7 Update lucene version to 7.6.0 2018-12-26 14:00:34 +08:00
magese 319f3a9f5e Update lucene version to 7.6.0 2018-12-26 13:59:28 +08:00
magese 0aa7d7d0b1 Update lucene version to 7.6.0 2018-12-26 13:58:46 +08:00
magese 93f58a6238 UPDATE README.md 2018-12-04 15:40:14 +08:00
magese cb6d72896d Update dict, Remove redundant dict; 2018-12-04 15:38:21 +08:00
magese 9403ef6dd7 UPDATE README.md 2018-12-04 15:35:27 +08:00
magese b83f833128 UPDATE README.md 2018-11-19 10:09:00 +08:00
magese 5c361fa330 UPDATE README.md 2018-11-15 14:52:55 +08:00
magese 304e978b91 UPDATE README.md 2018-11-15 14:52:03 +08:00
magese 67dc15cf74 UPDATE README.md 2018-11-15 14:23:33 +08:00
magese c61b0a1114 UPDATE README.md 2018-11-15 14:11:04 +08:00
magese c772277647 UPDATE README.md 2018-11-15 13:59:49 +08:00
magese 5a43f340f6 Delete Badge 2018-11-15 11:44:47 +08:00
magese 3e118d8073 Add Travis CI properties 2018-11-15 11:22:23 +08:00
magese 9618da0a11 Add Badges 2018-11-15 11:06:23 +08:00
magese 988caebe85 Update LICENSE 2018-11-15 11:05:24 +08:00
44 changed files with 2069819 additions and 474351 deletions
+38
View File
@@ -0,0 +1,38 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
+3
View File
@@ -0,0 +1,3 @@
language: java
jdk:
- oraclejdk8
+1
View File
@@ -0,0 +1 @@
+193
View File
@@ -0,0 +1,193 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
  10.You are not allowed to remove or modify any donation links/images in
this project, otherwise we will reclaim all authorization.
11.You're free to use this software at your own/you company/your orgnization 's
use. BUT You don't have the permission to sell the copy or the
redistribution of this software to others in any form.
12.If you've forked this project, you must use the same lisence.
END OF TERMS AND CONDITIONS
  Copyright 2018 Magese magese@live.cn
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+58
View File
@@ -0,0 +1,58 @@
## Solr-Cloud说明
##### 因为`Solr-Cloud`中的配置文件是交由`zookeeper`进行管理的, 所以为了方便更新动态词典, 所以也要将动态词典文件上传至`zookeeper`中,目录与`solr`的配置文件目录一致。
#### 注意:因为`zookeeper`中的配置文件大小不能超过`1m`,当词典列表过多时,需将词典文件切分成多个。
1. 将jar包放入每台服务器的Solr服务的`Jetty``Tomcat``webapp/WEB-INF/lib/`目录下;
2.`resources`目录下的`IKAnalyzer.cfg.xml``ext.dic``stopword.dic`放入solr服务的`Jetty``Tomcat``webapp/WEB-INF/classes/`目录下;
```console
① IKAnalyzer.cfg.xml (IK默认的配置文件,用于配置自带的扩展词典及停用词典)
② ext.dic (默认的扩展词典)
③ stopword.dic (默认的停词词典)
```
注意:与单机版不同,`ik.conf`及`dynamicdic.txt`请不要放在`classes`目录下!
3. 将`resources`目录下的`ik.conf`及`dynamicdic.txt`放入solr配置文件夹中,与solr的`managed-schema`文件同目录中;
```console
① ik.conf (动态词典配置文件)
files (动态词典列表,可以设置多个词典表,用逗号进行分隔,默认动态词典表为dynamicdic.txt)
lastupdate (默认值为0,每次对动态词典表修改后请修改该值,必须大于上次的值,不然不会将词典表中新的词语添加到内存中。)
② dynamicdic.txt (默认的动态词典,在此文件配置的词语不需重启服务即可加载进内存中。以#开头的词语视为注释,将不会加载到内存中。)
```
4. 配置Solr的`managed-schema`,添加`ik分词器`,示例如下;
```console
<!-- ik分词器 -->
<fieldType name="text_ik" class="solr.TextField">
<analyzer type="index">
<tokenizer class="org.wltea.analyzer.lucene.IKTokenizerFactory" useSmart="false" conf="ik.conf"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="org.wltea.analyzer.lucene.IKTokenizerFactory" useSmart="true" conf="ik.conf"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
```
5. 将配置文件上传至`zookeeper`中,首次使用请重启服务或reload Collection。
6. 测试分词:
* 此时的动态词典文件为空
![](./img/dic.png)
* 配置文件lastupdate为0
![](./img/conf.png)
* 测试分词
![](./img/cloud-analyzer.png)
7. 测试动态词典:
* 增加动态词典词语并上传至`zookeeper`
![](./img/dic2.png)
* 修改配置文件并上传至`zookeeper`
![](./img/conf2.png)
* 测试分词
![](./img/cloud-analyzer2.png)
+121 -77
View File
@@ -1,84 +1,128 @@
# ik-analyzer-solr7
ik-analyzer for solr7.x
<p>IKAnalyzer的原作者为林良益(linliangyi2007@gmail.com),项目网站为http://code.google.com/p/ik-analyzer/</p>
# ik-analyzer-solr
ik-analyzer for solr 7.x-8.x
* 该项目根据博主[@星火燎原智勇](http://www.cnblogs.com/liang1101/articles/6395016.html)的博客进行修改,其GITHUB地址为[@liang68](https://github.com/liang68)
<!-- Badges section here. -->
[![GitHub version](https://img.shields.io/maven-central/v/com.github.magese/ik-analyzer.svg?style=flat-square)](https://search.maven.org/search?q=g:com.github.magese%20AND%20a:ik-analyzer&core=gav)
[![Open Source Love](https://badges.frapsoft.com/os/v1/open-source.svg?v=103)](https://github.com/magese/ik-analyzer-solr/releases)
[![Crates.io](https://img.shields.io/crates/l/rustc-serialize.svg)](./LICENSE)
[![Build Status](https://travis-ci.org/magese/ik-analyzer-solr.svg?branch=master)](https://travis-ci.org/magese/ik-analyzer-solr)
[![HitCount](http://hits.dwyl.io/magese/ik-analyzer-solr.svg)](http://hits.dwyl.io/magese/ik-analyzer-solr)
<h4>适配最新版solr7,并添加动态加载字典表功能;</h4>
<h4>在不需要重启solr服务的情况下加载新增的字典。</h4>
[![GitHub forks](https://img.shields.io/github/forks/magese/ik-analyzer-solr.svg?style=social&label=Fork)](https://github.com/magese/ik-analyzer-solr/network/members)
[![GitHub stars](https://img.shields.io/github/stars/magese/ik-analyzer-solr.svg?style=social&label=Star)](https://github.com/magese/ik-analyzer-solr/stargazers)
<!-- /Badges section end. -->
>更新说明
>* 2018-10-10: 升级lucene版本为7.5.0
>* 2018-09-03: 优化注释与输出信息,取消部分中文输出避免不同字符集乱码,现会打印被调用inform方法的hashcode
>* 2018-08-23:
<br>&nbsp;&nbsp;⑴完善了动态更新词库代码注释;
<br>&nbsp;&nbsp;⑵将ik.conf配置文件中的lastUpdate属性改为long类型,现已支持时间戳形式
>* 2018-08-13: 更新maven仓库地址
>* 2018-08-01: 移除默认的扩展词与停用词
>* 2018-07-23: 升级lucene版本为7.4.0
## 简介
#### 适配最新版本solr 7&8
#### 扩展IK原有词库:
| 分词工具 | 词库中词的数量 | 最后更新时间 |
| :------: | :------: | :------: |
| ik | 27.5万 | 2012年 |
| mmseg | 15.7万 | 2017年 |
| word | 64.2万 | 2014年 |
| jieba | 58.4万 | 2012年 |
| jcesg | 16.6万 | 2018年 |
| sougou词库 | 115.2万 | 2019年 |
#### 将以上词库进行整理后约187.1万条词汇;
#### 添加动态加载词典表功能,在不需要重启solr服务的情况下加载新增的词典。
* IKAnalyzer的原作者为林良益<linliangyi2007@gmail.com>,项目网站为<http://code.google.com/p/ik-analyzer>
* 该项目动态加载功能根据博主[@星火燎原智勇](http://www.cnblogs.com/liang1101/articles/6395016.html)的博客进行修改,其GITHUB地址为[@liang68](https://github.com/liang68)
<hr>
<H2>使用说明:</H2><BR>
* jar包下载地址:[ik-analyzer-7.5.0.jar](https://search.maven.org/remotecontent?filepath=com/github/magese/ik-analyzer/7.5.0/ik-analyzer-7.5.0.jar)
* 历史版本:[Central Repository](https://search.maven.org/search?q=g:com.github.magese%20AND%20a:ik-analyzer&core=gav)
## 使用说明
* jar包下载地址:[![GitHub version](https://img.shields.io/badge/version-8.1.1-519dd9.svg)](https://search.maven.org/remotecontent?filepath=com/github/magese/ik-analyzer/8.1.1/ik-analyzer-8.1.1.jar)
* 历史版本:[![GitHub version](https://img.shields.io/maven-central/v/com.github.magese/ik-analyzer.svg?style=flat-square)](https://search.maven.org/search?q=g:com.github.magese%20AND%20a:ik-analyzer&core=gav)
<pre>
&lt;!-- Maven仓库地址 --&gt;
&lt;dependency&gt;
&lt;groupId&gt;com.github.magese&lt;/groupId&gt;
&lt;artifactId&gt;ik-analyzer&lt;/artifactId&gt;
&lt;version&gt;7.5.0&lt;/version&gt;
&lt;/dependency&gt;
</pre>
<ul>
<li>
<p>1. 将jar包放入solr服务的jetty或tomcat的webapp/WEB-INF/lib/目录下;</p>
</li>
<li>
<p>2. 将resources目录下的5个配置文件放入solr服务的jetty或tomcat的webapp/WEB-INF/classes/目录下;</p>
<pre>
①IKAnalyzer.cfg.xml
②ext.dic
③stopword.dic
④ik.conf
⑤dynamicdic.txt
</pre>
</li>
<li>
<p>3. 配置solr的managed-schema,添加ik分词器,示例如下;</p>
<pre>
&lt;!-- ik分词器 --&gt;
&lt;fieldType name="text_ik" class="solr.TextField"&gt;
&lt;analyzer type="index"&gt;
&lt;tokenizer class="org.wltea.analyzer.lucene.IKTokenizerFactory" useSmart="false" conf="ik.conf"/&gt;
&lt;filter class="solr.LowerCaseFilterFactory"/&gt;
&lt;/analyzer&gt;
&lt;analyzer type="query"&gt;
&lt;tokenizer class="org.wltea.analyzer.lucene.IKTokenizerFactory" useSmart="true" conf="ik.conf"/&gt;
&lt;filter class="solr.LowerCaseFilterFactory"/&gt;
&lt;/analyzer&gt;
&lt;/fieldType&gt;
</pre>
</li>
<li>
<p>4. 启动solr服务测试分词;</p>
</li>
<li>
<p>5. ik.conf文件说明:</p>
<pre>
files=dynamicdic.txt
lastupdate=0
</pre>
<p>files为动态字典列表,可以设置多个字典表,用逗号进行分隔,默认动态字典表为dynamicdic.txt</p>
<p>lastupdate默认值为0,每次对动态字典表修改后请+1,不然不会将字典表中新的词语添加到内存中。<s>lastupdate采用的是int类型,不支持时间戳,如果使用时间戳的朋友可以把源码中的int改成long即可;</s></p>
<p>2018-08-23 已将源码中lastUpdate改为long类型,现可以用时间戳了。</p>
</li>
<li>
<p>5-dynamicdic.txt 为动态字典,在此文件配置的词语不需重启服务即可加载进内存中;</p>
</li>
</ul>
<hr>
```console
<!-- Maven仓库地址 -->
<dependency>
<groupId>com.github.magese</groupId>
<artifactId>ik-analyzer</artifactId>
<version>8.1.1</version>
</dependency>
```
<p>有问题可以联系作者邮箱magese@live.cn</p>
<p>欢迎大家一起交流~</p>
### Solr-Cloud
* [Solr-Cloud说明](./README-CLOUD.md)
### 单机版Solr
1. 将jar包放入Solr服务的`Jetty`或`Tomcat`的`webapp/WEB-INF/lib/`目录下;
2. 将`resources`目录下的5个配置文件放入solr服务的`Jetty`或`Tomcat`的`webapp/WEB-INF/classes/`目录下;
```console
① IKAnalyzer.cfg.xml
② ext.dic
③ stopword.dic
④ ik.conf
⑤ dynamicdic.txt
```
3. 配置Solr的`managed-schema`,添加`ik分词器`,示例如下;
```console
<!-- ik分词器 -->
<fieldType name="text_ik" class="solr.TextField">
<analyzer type="index">
<tokenizer class="org.wltea.analyzer.lucene.IKTokenizerFactory" useSmart="false" conf="ik.conf"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="org.wltea.analyzer.lucene.IKTokenizerFactory" useSmart="true" conf="ik.conf"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
```
4. 启动Solr服务测试分词;
![analyzer](./img/analyzer.png)
5. `ik.conf`文件说明:
```console
files=dynamicdic.txt
lastupdate=0
```
1. `files`为动态词典列表,可以设置多个词典表,用逗号进行分隔,默认动态词典表为`dynamicdic.txt`
2. `lastupdate`默认值为`0`,每次对动态词典表修改后请+1,不然不会将词典表中新的词语添加到内存中。<s>`lastupdate`采用的是`int`类型,不支持时间戳,如果使用时间戳的朋友可以把源码中的`int`改成`long`即可;</s> `2018-08-23` 已将源码中`lastUpdate`改为`long`类型,现可以用时间戳了。
6. `dynamicdic.txt` 为动态词典
在此文件配置的词语不需重启服务即可加载进内存中。
以`#`开头的词语视为注释,将不会加载到内存中。
## 更新说明
- `2019-07-11:` 升级lucene版本为`8.1.1`
- `2019-05-27:`
- 升级lucene版本为`8.1.0`
- 优化原词典部分重复词语
- 更新搜狗2019最新流行词汇词典,约20k词汇量
- `2019-05-15:` 升级lucene版本为`8.0.0`,并支持Solr8使用
- `2019-03-01:` 升级lucene版本为`7.7.1`
- `2019-02-15:` 升级lucene版本为`7.7.0`
- `2018-12-26:`
- 升级lucene版本为`7.6.0`
- 兼容solr-cloud,动态词典配置文件及动态词典可交由`zookeeper`进行管理
- 动态词典增加注释功能,以`#`开头的行将视为注释
- `2018-12-04:` 整理更新词库列表`magese.dic`
- `2018-10-10:` 升级lucene版本为`7.5.0`
- `2018-09-03:` 优化注释与输出信息,取消部分中文输出避免不同字符集乱码,现会打印被调用inform方法的hashcode
- `2018-08-23: `
- 完善了动态更新词库代码注释;
- 将ik.conf配置文件中的lastUpdate属性改为long类型,现已支持时间戳形式
- `2018-08-13:` 更新maven仓库地址
- `2018-08-01:` 移除默认的扩展词与停用词
- `2018-07-23:` 升级lucene版本为`7.4.0`
## 感谢 Thanks
[![IDEA](./img/jetbrains.svg)](https://www.jetbrains.com/?from=ik-analyzer-solr)
[![forthebadge](https://forthebadge.com/images/badges/made-with-java.svg)](https://www.java.com)
## BUG & 疑问 & 其它
如果您在使用过程中遇到了BUG,或者有不清楚的地方,请挂ISSUE或者联系作者:<magese@live.cn>
如果您觉得该项目对您有帮助,请别忘记给这个项目一个`star`
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 84 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

+66
View File
@@ -0,0 +1,66 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 19.1.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
width="120.1px" height="130.2px" viewBox="0 0 120.1 130.2" style="enable-background:new 0 0 120.1 130.2;" xml:space="preserve"
>
<g>
<linearGradient id="XMLID_2_" gradientUnits="userSpaceOnUse" x1="31.8412" y1="120.5578" x2="110.2402" y2="73.24">
<stop offset="0" style="stop-color:#FCEE39"/>
<stop offset="1" style="stop-color:#F37B3D"/>
</linearGradient>
<path id="XMLID_3041_" style="fill:url(#XMLID_2_);" d="M118.6,71.8c0.9-0.8,1.4-1.9,1.5-3.2c0.1-2.6-1.8-4.7-4.4-4.9
c-1.2-0.1-2.4,0.4-3.3,1.1l0,0l-83.8,45.9c-1.9,0.8-3.6,2.2-4.7,4.1c-2.9,4.8-1.3,11,3.6,13.9c3.4,2,7.5,1.8,10.7-0.2l0,0l0,0
c0.2-0.2,0.5-0.3,0.7-0.5l78-54.8C117.3,72.9,118.4,72.1,118.6,71.8L118.6,71.8L118.6,71.8z"/>
<linearGradient id="XMLID_3_" gradientUnits="userSpaceOnUse" x1="48.3607" y1="6.9083" x2="119.9179" y2="69.5546">
<stop offset="0" style="stop-color:#EF5A6B"/>
<stop offset="0.57" style="stop-color:#F26F4E"/>
<stop offset="1" style="stop-color:#F37B3D"/>
</linearGradient>
<path id="XMLID_3049_" style="fill:url(#XMLID_3_);" d="M118.8,65.1L118.8,65.1L55,2.5C53.6,1,51.6,0,49.3,0
c-4.3,0-7.7,3.5-7.7,7.7v0c0,2.1,0.8,3.9,2.1,5.3l0,0l0,0c0.4,0.4,0.8,0.7,1.2,1l67.4,57.7l0,0c0.8,0.7,1.8,1.2,3,1.3
c2.6,0.1,4.7-1.8,4.9-4.4C120.2,67.3,119.7,66,118.8,65.1z"/>
<linearGradient id="XMLID_4_" gradientUnits="userSpaceOnUse" x1="52.9467" y1="63.6407" x2="10.5379" y2="37.1562">
<stop offset="0" style="stop-color:#7C59A4"/>
<stop offset="0.3852" style="stop-color:#AF4C92"/>
<stop offset="0.7654" style="stop-color:#DC4183"/>
<stop offset="0.957" style="stop-color:#ED3D7D"/>
</linearGradient>
<path id="XMLID_3042_" style="fill:url(#XMLID_4_);" d="M57.1,59.5C57,59.5,17.7,28.5,16.9,28l0,0l0,0c-0.6-0.3-1.2-0.6-1.8-0.9
c-5.8-2.2-12.2,0.8-14.4,6.6c-1.9,5.1,0.2,10.7,4.6,13.4l0,0l0,0C6,47.5,6.6,47.8,7.3,48c0.4,0.2,45.4,18.8,45.4,18.8l0,0
c1.8,0.8,3.9,0.3,5.1-1.2C59.3,63.7,59,61,57.1,59.5z"/>
<linearGradient id="XMLID_5_" gradientUnits="userSpaceOnUse" x1="52.1736" y1="3.7019" x2="10.7706" y2="37.8971">
<stop offset="0" style="stop-color:#EF5A6B"/>
<stop offset="0.364" style="stop-color:#EE4E72"/>
<stop offset="1" style="stop-color:#ED3D7D"/>
</linearGradient>
<path id="XMLID_3057_" style="fill:url(#XMLID_5_);" d="M49.3,0c-1.7,0-3.3,0.6-4.6,1.5L4.9,28.3c-0.1,0.1-0.2,0.1-0.2,0.2l-0.1,0
l0,0c-1.7,1.2-3.1,3-3.9,5.1C-1.5,39.4,1.5,45.9,7.3,48c3.6,1.4,7.5,0.7,10.4-1.4l0,0l0,0c0.7-0.5,1.3-1,1.8-1.6l34.6-31.2l0,0
c1.8-1.4,3-3.6,3-6.1v0C57.1,3.5,53.6,0,49.3,0z"/>
<g id="XMLID_3008_">
<rect id="XMLID_3033_" x="34.6" y="37.4" style="fill:#000000;" width="51" height="51"/>
<rect id="XMLID_3032_" x="39" y="78.8" style="fill:#FFFFFF;" width="19.1" height="3.2"/>
<g id="XMLID_3009_">
<path id="XMLID_3030_" style="fill:#FFFFFF;" d="M38.8,50.8l1.5-1.4c0.4,0.5,0.8,0.8,1.3,0.8c0.6,0,0.9-0.4,0.9-1.2l0-5.3l2.3,0
l0,5.3c0,1-0.3,1.8-0.8,2.3c-0.5,0.5-1.3,0.8-2.3,0.8C40.2,52.2,39.4,51.6,38.8,50.8z"/>
<path id="XMLID_3028_" style="fill:#FFFFFF;" d="M45.3,43.8l6.7,0v1.9l-4.4,0V47l4,0l0,1.8l-4,0l0,1.3l4.5,0l0,2l-6.7,0
L45.3,43.8z"/>
<path id="XMLID_3026_" style="fill:#FFFFFF;" d="M55,45.8l-2.5,0l0-2l7.3,0l0,2l-2.5,0l0,6.3l-2.3,0L55,45.8z"/>
<path id="XMLID_3022_" style="fill:#FFFFFF;" d="M39,54l4.3,0c1,0,1.8,0.3,2.3,0.7c0.3,0.3,0.5,0.8,0.5,1.4v0
c0,1-0.5,1.5-1.3,1.9c1,0.3,1.6,0.9,1.6,2v0c0,1.4-1.2,2.3-3.1,2.3l-4.3,0L39,54z M43.8,56.6c0-0.5-0.4-0.7-1-0.7l-1.5,0l0,1.5
l1.4,0C43.4,57.3,43.8,57.1,43.8,56.6L43.8,56.6z M43,59l-1.8,0l0,1.5H43c0.7,0,1.1-0.3,1.1-0.8v0C44.1,59.2,43.7,59,43,59z"/>
<path id="XMLID_3019_" style="fill:#FFFFFF;" d="M46.8,54l3.9,0c1.3,0,2.1,0.3,2.7,0.9c0.5,0.5,0.7,1.1,0.7,1.9v0
c0,1.3-0.7,2.1-1.7,2.6l2,2.9l-2.6,0l-1.7-2.5h-1l0,2.5l-2.3,0L46.8,54z M50.6,58c0.8,0,1.2-0.4,1.2-1v0c0-0.7-0.5-1-1.2-1
l-1.5,0v2H50.6z"/>
<path id="XMLID_3016_" style="fill:#FFFFFF;" d="M56.8,54l2.2,0l3.5,8.4l-2.5,0l-0.6-1.5l-3.2,0l-0.6,1.5l-2.4,0L56.8,54z
M58.8,59l-0.9-2.3L57,59L58.8,59z"/>
<path id="XMLID_3014_" style="fill:#FFFFFF;" d="M62.8,54l2.3,0l0,8.3l-2.3,0L62.8,54z"/>
<path id="XMLID_3012_" style="fill:#FFFFFF;" d="M65.7,54l2.1,0l3.4,4.4l0-4.4l2.3,0l0,8.3l-2,0L68,57.8l0,4.6l-2.3,0L65.7,54z"
/>
<path id="XMLID_3010_" style="fill:#FFFFFF;" d="M73.7,61.1l1.3-1.5c0.8,0.7,1.7,1,2.7,1c0.6,0,1-0.2,1-0.6v0
c0-0.4-0.3-0.5-1.4-0.8c-1.8-0.4-3.1-0.9-3.1-2.6v0c0-1.5,1.2-2.7,3.2-2.7c1.4,0,2.5,0.4,3.4,1.1l-1.2,1.6
c-0.8-0.5-1.6-0.8-2.3-0.8c-0.6,0-0.8,0.2-0.8,0.5v0c0,0.4,0.3,0.5,1.4,0.8c1.9,0.4,3.1,1,3.1,2.6v0c0,1.7-1.3,2.7-3.4,2.7
C76.1,62.5,74.7,62,73.7,61.1z"/>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 4.8 KiB

+7 -7
View File
@@ -4,16 +4,16 @@
<groupId>com.github.magese</groupId>
<artifactId>ik-analyzer</artifactId>
<version>7.5.0</version>
<version>8.1.1</version>
<packaging>jar</packaging>
<name>ik-analyzer-solr7</name>
<name>ik-analyzer-solr</name>
<url>http://code.google.com/p/ik-analyzer/</url>
<description>IK-Analyzer for solr7.5</description>
<description>IK-Analyzer for solr 7-8</description>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<lucene.version>7.5.0</lucene.version>
<lucene.version>8.1.1</lucene.version>
<javac.src.version>1.8</javac.src.version>
<javac.target.version>1.8</javac.target.version>
<maven.compiler.plugin.version>3.3</maven.compiler.plugin.version>
@@ -55,9 +55,9 @@
</licenses>
<scm>
<tag>master</tag>
<url>https://github.com/magese/ik-analyzer-solr7</url>
<connection>scm:git:git@github.com:magese/ik-analyzer-solr7.git</connection>
<developerConnection>scm:git:git@github.com:magese/ik-analyzer-solr7.git</developerConnection>
<url>https://github.com/magese/ik-analyzer-solr</url>
<connection>scm:git:git@github.com:magese/ik-analyzer-solr.git</connection>
<developerConnection>scm:git:git@github.com:magese/ik-analyzer-solr.git</developerConnection>
</scm>
<developers>
<developer>
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.cfg;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.cfg;
@@ -19,16 +41,16 @@ public class DefaultConfig implements Configuration {
/*
* 分词器默认字典路径
*/
private static final String PATH_DIC_MAIN = "org/wltea/analyzer/dic/main2012.dic";
private static final String PATH_DIC_QUANTIFIER = "org/wltea/analyzer/dic/quantifier.dic";
private static final String PATH_DIC_MAIN = "dict/magese.dic";
private static final String PATH_DIC_QUANTIFIER = "dict/quantifier.dic";
/*
* 分词器配置文件路径
*/
private static final String FILE_NAME = "IKAnalyzer.cfg.xml";
//配置属性——扩展字典
// 配置属性——扩展字典
private static final String EXT_DICT = "ext_dict";
//配置属性——扩展停止词典
// 配置属性——扩展停止词典
private static final String EXT_STOP = "ext_stopwords";
private Properties props;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -241,7 +263,7 @@ public class Lexeme implements Comparable<Lexeme>{
*
*/
public String toString(){
return String.valueOf(this.getBeginPosition()) + "-" + this.getEndPosition() +
return this.getBeginPosition() + "-" + this.getEndPosition() +
" : " + this.lexemeText + " : \t" +
this.getLexemeTypeString();
}
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.core;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.dic;
@@ -1,14 +1,33 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.dic;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.util.Collection;
import java.util.List;
@@ -91,32 +110,30 @@ public class Dictionary {
* 重新更新词典
* 由于停用词等不经常变也不建议常增加,故这里只修改动态扩展词库
*
* @param inputStreamList 词典文件IO流集合
* @param inputStreamReaderList 词典文件IO流集合
*/
public static void reloadDic(List<InputStream> inputStreamList) {
public static void reloadDic(List<Reader> inputStreamReaderList) {
// 如果本类单例尚未实例化,则先进行初始化操作
if (singleton == null) {
Configuration cfg = DefaultConfig.getInstance();
initial(cfg);
}
// 对词典流集合进行循环读取,将读取到的词语加载到主词典中
for (InputStream is : inputStreamList) {
for (Reader in : inputStreamReaderList) {
try {
BufferedReader br = new BufferedReader(new InputStreamReader(is, StandardCharsets.UTF_8), 512);
LineNumberReader br = new LineNumberReader(in);
String theWord;
do {
theWord = br.readLine();
if (theWord != null && !"".equals(theWord.trim())) {
singleton._MainDict.fillSegment(theWord.trim().toLowerCase().toCharArray());
}
} while (theWord != null);
while ((theWord = br.readLine()) != null) {
if (theWord.trim().length() == 0 || theWord.trim().charAt(0) == '#') continue;
singleton._MainDict.fillSegment(theWord.trim().toLowerCase().toCharArray());
}
} catch (IOException ioe) {
System.err.println("Other Dictionary loading exception.");
ioe.printStackTrace();
} finally {
try {
if (is != null) {
is.close();
if (in != null) {
in.close();
}
} catch (IOException e) {
e.printStackTrace();
+25 -3
View File
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.dic;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.lucene;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.lucene;
@@ -35,7 +57,7 @@ public final class IKTokenizer extends Tokenizer {
private int endPosition;
/**
* Lucene 7.4 Tokenizer适配器类构造函数
* Lucene 7.6 Tokenizer适配器类构造函数
*/
public IKTokenizer() {
this(false);
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.lucene;
@@ -14,6 +36,11 @@ import org.wltea.analyzer.dic.Dictionary;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.nio.charset.CharsetDecoder;
import java.nio.charset.CodingErrorAction;
import java.nio.charset.StandardCharsets;
import java.util.*;
/**
@@ -28,7 +55,9 @@ public class IKTokenizerFactory extends TokenizerFactory implements ResourceLoad
public IKTokenizerFactory(Map<String, String> args) {
super(args);
String useSmartArg = args.get("useSmart");
String confArg = args.get("conf");
this.setUseSmart(Boolean.parseBoolean(useSmartArg));
this.setConf(confArg);
}
@Override
@@ -45,10 +74,10 @@ public class IKTokenizerFactory extends TokenizerFactory implements ResourceLoad
*/
@Override
public void inform(ResourceLoader resourceLoader) throws IOException {
System.out.println(String.format("IKTokenizerFactory "+ this.hashCode() +" inform conf: %s", this.conf));
System.out.println(String.format("IKTokenizerFactory " + this.hashCode() + " inform conf: %s", getConf()));
this.loader = resourceLoader;
update();
if ((this.conf != null) && (!this.conf.trim().isEmpty())) {
if ((getConf() != null) && (!getConf().trim().isEmpty())) {
UpdateThread.getInstance().register(this);
}
}
@@ -60,24 +89,27 @@ public class IKTokenizerFactory extends TokenizerFactory implements ResourceLoad
*/
@Override
public void update() throws IOException {
// 默认UTF-8解码
CharsetDecoder decoder = StandardCharsets.UTF_8.newDecoder()
.onMalformedInput(CodingErrorAction.REPORT)
.onUnmappableCharacter(CodingErrorAction.REPORT);
// 获取ik.conf配置文件信息
Properties p = canUpdate();
if (p != null) {
// 获取词典表名称集合
List<String> dicPaths = SplitFileNames(p.getProperty("files"));
// 获取词典文件的IO流
List<InputStream> inputStreamList = new ArrayList<>();
List<Reader> inputStreamReaderList = new ArrayList<>();
for (String path : dicPaths) {
if ((path != null) && (!path.isEmpty())) {
InputStream is = this.loader.openResource(path);
if (is != null) {
inputStreamList.add(is);
}
Reader isr = new InputStreamReader(loader.openResource(path), decoder);
inputStreamReaderList.add(isr);
}
}
// 如果IO流集合不为空则执行加载词典
if (!inputStreamList.isEmpty())
Dictionary.reloadDic(inputStreamList);
if (!inputStreamReaderList.isEmpty())
Dictionary.reloadDic(inputStreamReaderList);
}
}
@@ -86,10 +118,10 @@ public class IKTokenizerFactory extends TokenizerFactory implements ResourceLoad
*/
private Properties canUpdate() {
try {
if (this.conf == null)
if (getConf() == null)
return null;
Properties p = new Properties();
InputStream confStream = this.loader.openResource(this.conf); // 获取配置文件流
InputStream confStream = this.loader.openResource(getConf()); // 获取配置文件流
p.load(confStream); // 读取配置文件
confStream.close(); // 关闭文件流
String lastupdate = p.getProperty("lastupdate", "0"); // 获取最后更新数字
@@ -100,13 +132,13 @@ public class IKTokenizerFactory extends TokenizerFactory implements ResourceLoad
String paths = p.getProperty("files"); // 获取词典文件名
if ((paths == null) || (paths.trim().isEmpty()))
return null;
System.out.println("loading ik.conf files success.");
System.out.println("loading " + getConf() + " files success.");
return p;
}
this.lastUpdateTime = t;
return null;
} catch (Exception e) {
System.err.println("parsing ik.conf NullPointerException!!!" + Arrays.toString(e.getStackTrace()));
System.err.println("parsing " + getConf() + " NullPointerException!!!" + Arrays.toString(e.getStackTrace()));
}
return null;
}
@@ -126,6 +158,7 @@ public class IKTokenizerFactory extends TokenizerFactory implements ResourceLoad
return result;
}
/* getter & setter */
private boolean useSmart() {
return useSmart;
}
@@ -133,4 +166,12 @@ public class IKTokenizerFactory extends TokenizerFactory implements ResourceLoad
private void setUseSmart(boolean useSmart) {
this.useSmart = useSmart;
}
private String getConf() {
return conf;
}
private void setConf(String conf) {
this.conf = conf;
}
}
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.lucene;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.query;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.query;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.sample;
@@ -1,7 +1,29 @@
/*
* IK 中文分词 版本 7.5
* IK Analyzer release 7.5
* update by Magese(magese@live.cn)
* IK 中文分词 版本 8.1.1
* IK Analyzer release 8.1.1
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* 源代码由林良益(linliangyi2005@gmail.com)提供
* 版权声明 2012,乌龙茶工作室
* provided by Linliangyi and copyright 2012 by Oolong studio
*
* 8.1.1版本 由 Magese (magese@live.cn) 更新
* release 8.1.1 update by Magese(magese@live.cn)
*
*/
package org.wltea.analyzer.sample;
@@ -17,10 +39,7 @@ import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.IndexWriterConfig.OpenMode;
import org.apache.lucene.queryparser.classic.ParseException;
import org.apache.lucene.queryparser.classic.QueryParser;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.search.*;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.RAMDirectory;
import org.wltea.analyzer.lucene.IKAnalyzer;
@@ -85,10 +104,11 @@ public class LuceneIndexAndSearchDemo {
//搜索相似度最高的5条记录
TopDocs topDocs = isearcher.search(query, 5);
System.out.println("命中:" + topDocs.totalHits);
long totalHits = topDocs.totalHits.value;
System.out.println("命中:" + totalHits);
//输出结果
ScoreDoc[] scoreDocs = topDocs.scoreDocs;
for (int i = 0; i < topDocs.totalHits; i++) {
for (int i = 0; i < totalHits; i++) {
Document targetDoc = isearcher.doc(scoreDocs[i].doc);
System.out.println("内容:" + targetDoc.toString());
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,316 +0,0 @@
世纪
位数
像素
克拉
公亩
公克
公分
公升
公尺
公担
公斤
公里
公顷
分钟
分米
加仑
千克
千米
厘米
周年
小时
平方
平方公尺
平方公里
平方分米
平方厘米
平方码
平方米
平方英寸
平方英尺
平方英里
平米
年代
年级
月份
毫升
毫米
毫克
海里
点钟
盎司
秒钟
立方公尺
立方分米
立方厘米
立方码
立方米
立方英寸
立方英尺
英亩
英寸
英尺
英里
阶段