10  Conclusion

This study investigated the interpretability and robustness of two widely-used explanation methods—LIME and SHAP—when applied to two contrasting models:

We aimed to assess whether model performance comes at the cost of interpretability, and whether explanation methods remain stable when the input text is perturbed. Below, we synthesize our findings around five core research questions.

10.1 Key Findings by Research Question

10.1.1 Explanation Consistency

  • SHAP consistently provides more stable and consistent explanations than LIME across both models.
  • SHAP + Logistic Regression had the most robust performance with a Jaccard similarity of 0.910, compared to 0.885 for SHAP + BERT.
  • LIME showed greater variability, especially on BERT (0.475), suggesting that contextual embeddings are harder to interpret reliably with local approximations.

10.1.2 Model Complexity vs. Interpretability

  • Logistic Regression produced clear, human-understandable explanations with both LIME and SHAP.
  • BERT, while more accurate, sacrificed interpretability. Its top words were often emotionally or contextually rich but not always intuitive.
  • The trade-off is clear: greater model capacity may obscure the path to interpretability.

10.1.3 Perturbation Stability

  • Explanation robustness was strongly affected by perturbations:
    • LIME + Logistic Regression: 0.662
    • LIME + BERT: 0.475
    • SHAP + Logistic Regression: 0.910
    • SHAP + BERT: 0.885
  • These results show that simpler models paired with SHAP yield more stable word-level explanations.

10.1.4 Accuracy vs. Explanation Quality

  • BERT achieved a ~23% increase in accuracy and outperformed Logistic Regression on all classification metrics.
  • However, this gain came with reduced explanation stability and lower transparency, especially under LIME.
  • In applications requiring both high performance and auditability, a combination of SHAP + simple models may offer a balanced solution.

10.1.5 Visualization as Interpretability Aid

  • Visual tools like bar plots and word clouds were essential for surfacing key tokens influencing model predictions.
  • SHAP’s additive attribution allowed for more consistent visualization across different inputs and models.
  • BERT explanations were harder to interpret visually due to their sensitivity to input semantics and syntax.

10.2 Final Takeaways

  • There exists a fundamental trade-off between model complexity and interpretability robustness.
  • SHAP outperforms LIME in explanation stability and reliability across both models.
  • Logistic Regression + SHAP offers the most trustworthy combination for interpretable NLP.
  • Practitioners should choose explanation tools and model types based on task sensitivity:
    • Use SHAP when explanation stability is essential.
    • Use LIME for local debugging and lightweight inspection.
    • Prefer simpler models in high-stakes or regulated contexts.

Conclusion: In socially sensitive NLP tasks such as sentiment analysis of public discourse, model transparency and explanation stability are not optional. Our study shows that while BERT improves classification performance, Logistic Regression—especially when interpreted with SHAP—offers the most dependable insight into model behavior.

10.3 Limitations & Future Work

10.3.1 Limitations

-Single-domain, single-language dataset: experiments use only English Twitter data.

-Limited model diversity: only logistic regression and BERT are evaluated.

-Restricted explanation methods: LIME and SHAP only.

-Narrow stability metric: Jaccard similarity ignores semantic equivalence.

10.3.2 Future Work

-Cross-domain & multilingual: test on legal, medical, non-English corpora.

-Expanded models & methods: include intermediate architectures (e.g. RNNs, DistilBERT).

-Enhanced stability metrics: add semantic-similarity measures (e.g. embedding cosine similarity).

-Interactive visualization: build a dashboard for real-time perturbation and comparison.