项目效果概览:

项目架构:

获取文件模板,前端就会展示问卷的列表,我们就可以选择选项,结果就会同步到我们的后端,通过后端把统计结果发生到前端,完成整个流程.

git仓库地址:

目录标题

项目效果概览:项目架构:git仓库地址:1.基础准备工作1.1lombok插件1.2maven依赖配置1.3 接口设计1.4模块式开发1.5common公共类1.5.1BaseResponseVO:实现的是公共返回值15.2CorsFilter:解决跨域问题 1.6utils工具类1.7conf配置读取类1.7.1配置application.yml文件1.7.2做一个测试的模板 2.微信调查问卷模板配置3.业务层实现(业务层实现在service包下)3.1接口WechatTemplateService,主要作用是提供controller进行调取的,真正实现方法在接口的实现类中3.2接口WechatTemplateService的实现类,通过此类实现接口中定义的方法 4.表现层controller实现5.程序业务测试6.Kafka Producer集成7.HTTPS的支持(因为微信小程序要求必须是HTTPS)7.1CA证书申请7.2域名绑定 8.集成SSL证书9.阿里云部署后端10.编译部署11.完整项目代码git仓库地址12.项目后续扩展12.1BigData实现

1.基础准备工作

1.1lombok插件

lomlok的作用主要是简化代码开发

1.2maven依赖配置

<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.2.5.RELEASE</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.imooc.jiangzh</groupId> <artifactId>kafka-study</artifactId> <version>0.0.1-SNAPSHOT</version> <name>kafka-study</name> <description>微信调查问卷</description> <properties> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <!--springBoot依赖两个---> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>cn.hutool</groupId> <artifactId>hutool-all</artifactId> <version>4.1.1</version> </dependency> <!--kafka相关依赖两个--> <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka-clients</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka-streams</artifactId> <version>2.4.0</version> </dependency> <!-- lombok依赖 --> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <version>1.18.12</version> <scope>provided</scope> </dependency> <dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>28.2-jre</version> </dependency> <!-- fastjson依赖 --> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.68</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>

1.3 接口设计

1.4模块式开发

成品样式:

1.5common公共类

1.5.1BaseResponseVO:实现的是公共返回值

用泛型去做枚举,当需要真实返回值的时候按照枚举类型进行传入就可以,当不需返回值的时候,只有RequestId

package com.yuge.wechat.questionnaire.common; import java.util.UUID; import lombok.Data; /** * @description : 公共返回对象 **/ @Data public class BaseResponseVO<M> { private String requestId; private M result; public static<M> BaseResponseVO success(){ BaseResponseVO baseResponseVO = new BaseResponseVO(); baseResponseVO.setRequestId(getRequestId()); return baseResponseVO; } public static<M> BaseResponseVO success(M result){ BaseResponseVO baseResponseVO = new BaseResponseVO(); baseResponseVO.setRequestId(getRequestId()); baseResponseVO.setResult(result); return baseResponseVO; } private static String getRequestId(){ return UUID.randomUUID().toString(); } }

15.2CorsFilter:解决跨域问题

package com.yuge.wechat.questionnaire.common; import java.io.IOException; import javax.servlet.Filter; import javax.servlet.FilterChain; import javax.servlet.ServletException; import javax.servlet.ServletRequest; import javax.servlet.ServletResponse; import javax.servlet.annotation.WebFilter; import javax.servlet.http.HttpServletResponse; import org.springframework.context.annotation.Configuration; /** * @description : 跨域问题解决 **/ @WebFilter(filterName = "CorsFilter") @Configuration public class CorsFilter implements Filter { @Override public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException { HttpServletResponse response = (HttpServletResponse) res; response.setHeader("Access-Control-Allow-Origin","*"); response.setHeader("Access-Control-Allow-Credentials", "true"); response.setHeader("Access-Control-Allow-Methods", "POST, GET, PATCH, DELETE, PUT"); response.setHeader("Access-Control-Max-Age", "3600"); response.setHeader("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept"); chain.doFilter(req, res); } }

1.6utils工具类

package com.yuge.wechat.questionnaire.utils; import com.alibaba.fastjson.JSON; import com.alibaba.fastjson.JSONArray; import com.alibaba.fastjson.JSONObject; import java.io.BufferedReader; import java.io.File; import java.io.FileNotFoundException; import java.io.FileReader; import java.io.IOException; import java.io.Reader; import java.util.Optional; import lombok.Cleanup; import lombok.extern.slf4j.Slf4j; /** * @description : 文件工具类 * 给一个路径filePath,返回路径下内容,重载了多种返回类型:string,JSONObject,JSONArray **/ @Slf4j public class FileUtils { public static String readFile(String filePath) throws IOException { @Cleanup BufferedReader reader = new BufferedReader( new FileReader(new File(filePath)) ); String lineStr = ""; StringBuffer stringBuffer = new StringBuffer(); while ((lineStr = reader.readLine()) != null) { stringBuffer.append(lineStr); } return stringBuffer.toString(); } public static Optional<JSONObject> readFile2JsonObject(String filePath){ try { String fileContent = readFile(filePath); log.info("readFile2Json fileContent: [{}]" , fileContent); return Optional.ofNullable(JSON.parseObject(fileContent)); } catch (IOException e) { e.printStackTrace(); } return Optional.empty(); } public static Optional<JSONArray> readFile2JsonArray(String filePath){ try { String fileContent = readFile(filePath); log.info("readFile2Json fileContent: [{}]" , fileContent); return Optional.ofNullable(JSON.parseArray(fileContent)); } catch (IOException e) { e.printStackTrace(); } return Optional.empty(); } }

1.7conf配置读取类

package com.yuge.wechat.questionnaire.conf; import lombok.Data; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.context.annotation.Configuration; import java.util.List; @Data @Configuration @ConfigurationProperties(prefix = "template") public class WechatTemplateProperties { private List<WetchTemplate> templates; private int templateResultType; //0-文件夹 1-数据库 private String templateResultFilePath;//结果文件路径 @Data public static class WetchTemplate{ private String templateId; //模板编号,用来模板对应接口的唯一标识 private String templateFilePath; private String active; } }

1.7.1配置application.yml文件

注意:conf类下定义的变量是驼峰命名templateResultType到application需要转成tempalte_result_type server: port: 8080 template: templates: - {"templateId":"1","templateFilePath":"C:/user/MyCode/wechat-questionnaire/src/main/resources/template/template.json","active":true} - {"templateId":"2","templateFilePath":"C:/user/MyCode/wechat-questionnaire/src/main/resources/template/template.json","active":false} template_result_type: 0 template_result_filePath: "C:/user/MyCode/wechat-questionnaire/src/main/resources/template/templateRestlt.json"

注意事项:路径是用反斜杠的,而反斜杠在java中是转义符所有把把反斜杠换成/才可以

这一个json是由多个数组组成的,一个数组就一个文件的题目和选项 前端展示的时候他会通过接口调这个文件的内容,文件是什么内容他就会展示什么内容

1.7.2做一个测试的模板

[{ "questionId": "1", //问题编号 "question": "今天几号", //问题内容 "answer": "", //默认答案 "options": [ //选项 {"label": "1 号", "value": "A"}, {"label": "2 号", "value": "B"}, {"label": "3 号", "value": "C"}, {"label": "4 号", "value": "D"} ]}, {"questionId": "2", "question": "你喜爱的颜色", "answer": "", "options": [ {"label": "红色", "value": "A"}, {"label": "黄色", "value": "B"}, {"label": "绿色", "value": "C"}, {"label": "紫色", "value": "D"} ]} ] { "templateId": "001", //模板编号 "totalNumber": "102", //有多少人作答 "statistics": [ { //针对这个templateId我们统计的结果是什么 "questionId": "1", "question": "今天几号", "answers": [ {"label": "A", "value": 10}, //选a的10人 ,选b的50人,选c的12人,选d的17人 {"label": "B", "value": 50}, {"label": "C", "value": 12}, {"label": "D", "value": 17} ] }, { "questionId": "2", "question": "你喜爱的颜色", "answers": [ {"label": "A", "value": 12}, {"label": "B", "value": 52}, {"label": "C", "value": 17}, {"label": "D", "value": 17} ] } ] }

一个存储模板一个存储模板的结果

2.微信调查问卷模板配置

3.业务层实现(业务层实现在service包下)

3.1接口WechatTemplateService,主要作用是提供controller进行调取的,真正实现方法在接口的实现类中

package com.yuge.wechat.questionnaire.service; import com.alibaba.fastjson.JSONObject; import com.yuge.wechat.questionnaire.conf.WechatTemplateProperties; public interface WechatTemplateservice { //前端获取统计的模板 获取active为true的模板 WechatTemplateProperties.WetchTemplate getWeChatQuestionnaireTemplate(); //前端返回调查问卷结果数据到后端 void reportTheResultsOfTheQuestionnaire(JSONObject resultsData); //后端把统计结果数据发送给前端 JSONObject questionnaireStatistics(String templateId); }

3.2接口WechatTemplateService的实现类,通过此类实现接口中定义的方法

package com.yuge.wechat.questionnaire.service; import com.alibaba.fastjson.JSONObject; import com.yuge.wechat.questionnaire.conf.WechatTemplateProperties; import com.yuge.wechat.questionnaire.utils.FileUtils; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import java.util.List; import java.util.Optional; @Slf4j //lombok提供的日志 @Service public class WechatTemplateServiceImpl implements WechatTemplateservice { @Autowired WechatTemplateProperties properties; /** * 前端向后端获取微信调查问卷为active模板 * @return */ @Override public WechatTemplateProperties.WetchTemplate getWeChatQuestionnaireTemplate() { List<WechatTemplateProperties.WetchTemplate> templates = properties.getTemplates(); Optional<WechatTemplateProperties.WetchTemplate> wetchTemplate = templates.stream().filter((template) -> template.isActive()).findFirst(); return wetchTemplate.isPresent()? wetchTemplate.get():null; } /** * 前端问卷结果返回后端 * @param resultsData */ @Override public void reportTheResultsOfTheQuestionnaire(JSONObject resultsData) { //kafka Producer 将数据推倒topic log.info("reportTheResultsOfTheQuestionnaire:[{}]",resultsData); } /** * 后端前前端发送调查问卷统计结果 * @param templateId * @return */ @Override public JSONObject questionnaireStatistics(String templateId) { if (templateId=="0"){ //0就文件获取 return FileUtils.readFile2JsonObject(properties.getTemplateResultFilePath()).get(); }else{ //其他数据源获取 return null; } } }

4.表现层controller实现

实现的是三个接口逻辑

package com.yuge.wechat.questionnaire.controller; import com.alibaba.fastjson.JSON; import com.alibaba.fastjson.JSONArray; import com.alibaba.fastjson.JSONObject; import com.yuge.wechat.questionnaire.common.BaseResponseVO; import com.yuge.wechat.questionnaire.conf.WechatTemplateProperties; import com.yuge.wechat.questionnaire.service.WechatTemplateservice; import com.yuge.wechat.questionnaire.utils.FileUtils; import org.apache.kafka.common.protocol.types.Field; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Controller; import org.springframework.web.bind.annotation.*; import java.util.HashMap; @Controller @RequestMapping("/v1") public class WechatTemplateController { @Autowired WechatTemplateservice wechatTemplateservice; @Autowired WechatTemplateProperties properties; /** * 获取调查问卷模板的数据(前端获取一个问卷内容,供客户来填写) */ @RequestMapping(value = "/getTheQuestionnaireTemplate",method = RequestMethod.GET) public BaseResponseVO getTheQuestionnaireTemplate(){ WechatTemplateProperties.WetchTemplate wetchTemplate = wechatTemplateservice.getWeChatQuestionnaireTemplate(); HashMap<String, Object> map = new HashMap<>(); map.put("templateId",wetchTemplate.getTemplateId()); map.put("template", FileUtils.readFile2JsonArray(properties.getTemplateResultFilePath())); return BaseResponseVO.success(map); } /** * 返回到后端的问卷填写数据, */ @RequestMapping(value = "/returnQuestionnaireResults",method = RequestMethod.POST) public BaseResponseVO returnQuestionnaireResults(@RequestBody String resultData){ wechatTemplateservice.reportTheResultsOfTheQuestionnaire(JSON.parseObject(resultData)); return BaseResponseVO.success(); } /** * 返回数据的统计结果,允许前端传入模板编号 */ @RequestMapping(value = "/getQuestionnaireStatistics",method = RequestMethod.GET) public BaseResponseVO getQuestionnaireStatistics(@RequestParam(value="templateId",required = false) String templateId){ JSONObject statistics = wechatTemplateservice.questionnaireStatistics(templateId); return BaseResponseVO.success(statistics); } }

5.程序业务测试

-- 查询模板信息 curl -XGET http://localhost:8080/v1/template -- 查询模板统计结果 curl -XGET http://localhost:8080/v1/template/result -- 传入调查问卷结果 curl -XPOST -H "Content-Type:application/json; charset=UTF-8" http://localhost:8080/v1/template/report -d \ '{ templateId:"001", result:[ {"questionId":"1","question":"今天几号","answer":"A"}, {"questionId":"2","question":"你喜爱的颜色","answer":"B"} ] }' netstat -ano | findstr 443

6.Kafka Producer集成

package com.imooc.jiangzh.kafka.wechat.conf; import lombok.Data; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.context.annotation.Configuration; @Data @Configuration @ConfigurationProperties(prefix = "wechat.kafka") public class KafkaProperties { private String bootstrapServers; private String acksConfig; private String retriesConfig; private String batchSizeConfig; private String lingerMsConfig; private String bufferMemoryConfig; private String keySerializerClassConfig; private String valueSerializerClassConfig; } package com.imooc.jiangzh.kafka.wechat.conf; import org.apache.kafka.clients.producer.KafkaProducer; import org.apache.kafka.clients.producer.Producer; import org.apache.kafka.clients.producer.ProducerConfig; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import java.util.Properties; @Configuration public class KafkaConf { @Autowired private KafkaProperties kafkaProperties; @Bean//增加@Bean注释:依赖spring提供的上下文,来做一个单例模式,这样我们所以的线程都是共享同一个kafkaProducer public Producer kafkaProducer(){ Properties properties = new Properties(); properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaProperties.getBootstrapServers()); properties.put(ProducerConfig.ACKS_CONFIG, kafkaProperties.getAcksConfig()); properties.put(ProducerConfig.RETRIES_CONFIG,kafkaProperties.getRetriesConfig()); properties.put(ProducerConfig.BATCH_SIZE_CONFIG,kafkaProperties.getBatchSizeConfig()); properties.put(ProducerConfig.LINGER_MS_CONFIG,kafkaProperties.getLingerMsConfig()); properties.put(ProducerConfig.BUFFER_MEMORY_CONFIG,kafkaProperties.getBufferMemoryConfig()); properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,kafkaProperties.getKeySerializerClassConfig()); properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,kafkaProperties.getValueSerializerClassConfig()); // Producer的主对象 Producer<String,String> producer = new KafkaProducer<>(properties); return producer; } }

7.HTTPS的支持(因为微信小程序要求必须是HTTPS)

7.1CA证书申请

7.2域名绑定

8.集成SSL证书

9.阿里云部署后端

10.编译部署

11.完整项目代码git仓库地址

12.项目后续扩展

上述只是简单的实现了微信小程序的调查问卷功能: 后期项目迭代增加需求: 1.通过mybaits依赖增加mysql数据库支持 2.增加大数据分析大屏数据展示

12.1BigData实现

我们通过kafkaProducer把数据发送到topic上面,但是数据并未做任何分析和展示,后期我们需要做一个数据大屏,来统计分析文件结果,并直观展示.

架构:kafkaProducer->consumer->hbase->sparkSQL->hbase->Kylin(数据展示)